2024-04-26T20:44:57.342 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:44:57.342 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/orig.config.yaml 2024-04-26T20:44:57.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:44:57.494 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7675181' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJnsyq10CpgBG9NopjK2nAvmIRB9eSdijA01lrUCzRX0grLO+FjWiFQP10X9XPBtgZ26sh1dXu35gYk2iDcB4RI= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ3naZqssDWUlcffq/JAsM8QYKsDmGsRKb64ZG6U42iUihoJ23dwRpZQEoBdfD+JU44wftFDpm4PSqoSebESaM= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGrC3jZJkjZOehWDiuNlsJT8N0Pz7YI8c5rfEUgeHgXFQEyOFzjenwzLQEUG1DXupChV/hnm6loDhEtF8P1mOAQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:44:57.494 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:44:57.495 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:44:57.495 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:44:57.496 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:44:57.519 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:44:57.520 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:44:57.520 INFO:teuthology.packaging:ref: None 2024-04-26T20:44:57.520 INFO:teuthology.packaging:tag: None 2024-04-26T20:44:57.520 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:44:57.520 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:44:57.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:44:57.663 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:44:57.664 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:44:57.671 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:44:57.671 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:44:57.698 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:44:57.719 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:44:57.725 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:44:57.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:37.407314', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJnsyq10CpgBG9NopjK2nAvmIRB9eSdijA01lrUCzRX0grLO+FjWiFQP10X9XPBtgZ26sh1dXu35gYk2iDcB4RI='} 2024-04-26T20:44:57.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:37.409304', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ3naZqssDWUlcffq/JAsM8QYKsDmGsRKb64ZG6U42iUihoJ23dwRpZQEoBdfD+JU44wftFDpm4PSqoSebESaM='} 2024-04-26T20:44:57.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:37.411035', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGrC3jZJkjZOehWDiuNlsJT8N0Pz7YI8c5rfEUgeHgXFQEyOFzjenwzLQEUG1DXupChV/hnm6loDhEtF8P1mOAQ='} 2024-04-26T20:44:57.786 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:44:57.792 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T20:44:57.792 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T20:44:57.792 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T20:44:57.793 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:44:57.924 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f08e411f160>, signals=[15]) 2024-04-26T20:44:57.925 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:44:57.931 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:44:57.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T20:44:57.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:44:58.010 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T20:44:58.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:44:58.086 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T20:44:58.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:44:58.162 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:44:58.169 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-04-26T20:44:58.187 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-04-26T20:44:58.187 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-04-26T20:44:58.244 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-04-26T20:44:58.244 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:44:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:44:58.246 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-04-26T20:44:58.270 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-26T20:44:58.288 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-26T20:44:58.288 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:44:58.345 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:44:58.346 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:44:58.346 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:44:58.346 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-26T20:44:58.370 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-04-26T20:44:58.388 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-04-26T20:44:58.388 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-04-26T20:44:58.445 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-04-26T20:44:58.445 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-04-26T20:44:58.445 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-04-26T20:44:58.445 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:44:58.446 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:44:58.446 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-04-26T20:44:58.470 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:44:58.480 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:44:58.537 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:44:58.538 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:44:58.540 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:44:58.543 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:44:58.559 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:44:58.572 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:44:58.573 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:44:58.598 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:44:58.600 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:44:58.696 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:44:58.778 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:44:58.792 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:44:58.792 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T20:44:58.793 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:44:58.793 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:44:58.794 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:44:58.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:44:58.810 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.811 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:44:58.811 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-26T20:44:58.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:44:58.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:44:58.813 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.813 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:44:58.813 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-26T20:44:58.813 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.813 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:44:58.814 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-26T20:44:58.868 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:44:58.869 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.869 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:44:58.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.874 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:44:58.874 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:44:58.874 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.874 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:44:58.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.875 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:58.875 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:44:58.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:59.016 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum upgrade -y linux-firmware 2024-04-26T20:44:59.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum upgrade -y linux-firmware 2024-04-26T20:44:59.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2024-04-26T20:45:01.412 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T20:45:01.474 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T20:45:01.658 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.0 MB 00:01 2024-04-26T20:45:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T20:45:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-04-26T20:45:06.249 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-04-26T20:45:07.560 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-26T20:45:09.961 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-26T20:45:10.299 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-26T20:45:11.906 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-26T20:45:13.427 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T20:45:14.129 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 21 MB 00:04 2024-04-26T20:45:17.224 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 866 kB/s | 6.2 MB 00:07 2024-04-26T20:45:18.749 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-04-26T20:45:18.899 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-26T20:45:19.187 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-26T20:45:19.970 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T20:45:20.764 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-26T20:45:20.765 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-26T20:45:20.765 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-26T20:45:20.879 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:45:20.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:45:21.104 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:45:21.105 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:45:21.107 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2024-04-26T20:45:21.114 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-26T20:45:21.115 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-26T20:45:21.115 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-26T20:45:21.227 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:45:21.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:45:21.373 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:45:21.374 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:45:21.376 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-04-26T20:45:25.176 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 9.3 kB/s | 1.7 kB 00:00 2024-04-26T20:45:27.159 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-04-26T20:45:27.160 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-04-26T20:45:27.161 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-04-26T20:45:27.282 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T20:45:27.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:45:27.432 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:45:27.432 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:45:27.434 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-04-26T20:45:38.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26 20:45:38 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:45:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2024-04-26 20:45:38-- 2024-04-26T20:45:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 18s 2024-04-26T20:45:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 341M in 17s (19.7 MB/s) 2024-04-26T20:45:38.670 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:45:38.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:45:40.866 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2024-04-26T20:45:40.999 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2024-04-26T20:45:41.120 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2024-04-26T20:45:49.392 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:45:49.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26 20:45:49 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:45:49.969 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-04-26 20:45:49-- 2024-04-26T20:45:49.969 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 23s 2024-04-26T20:45:49.969 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 341M in 22s (15.8 MB/s) 2024-04-26T20:45:49.972 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T20:45:49.972 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:45:52.107 INFO:teuthology.orchestra.run.smithi204.stdout:Verifying... ######################################## 2024-04-26T20:45:52.239 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing... ######################################## 2024-04-26T20:45:52.362 INFO:teuthology.orchestra.run.smithi204.stdout:Updating / installing... 2024-04-26T20:45:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26 20:45:58 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:45:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-04-26 20:45:58-- 2024-04-26T20:45:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 37s 2024-04-26T20:45:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 341M in 36s (9.35 MB/s) 2024-04-26T20:45:58.340 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:45:58.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:46:00.306 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:46:00.419 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying... ######################################## 2024-04-26T20:46:00.535 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing... ######################################## 2024-04-26T20:46:00.639 INFO:teuthology.orchestra.run.smithi094.stdout:Updating / installing... 2024-04-26T20:46:08.553 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:46:29.232 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:46:29.232 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:46:29.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:46:29.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:46:29.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:46:29.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:46:29.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:46:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:46:29.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:46:29.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:46:29.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:46:29.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:46:29.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:46:29.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:46:29.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:46:29.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:46:29.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:46:29.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:46:29.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:46:29.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:46:29.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:46:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:46:29.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:46:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:46:29.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:46:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:46:29.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:46:29.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:46:29.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:46:29.355 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:46:29.356 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:46:29.357 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:46:29.358 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:46:29.359 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:46:29.360 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:46:29.361 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:46:29.362 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:46:29.363 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:46:29.364 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:46:29.365 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:46:29.366 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:46:29.367 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:46:29.368 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:46:29.369 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:46:29.370 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:46:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:46:29.372 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:46:29.373 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:46:29.374 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:46:29.375 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:46:29.376 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:46:29.377 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:46:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:46:29.379 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:46:29.380 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:46:29.381 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:46:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:46:29.383 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:46:29.384 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:46:29.385 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:46:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:46:29.387 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:46:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:46:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:46:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:46:29.391 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:46:29.392 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:46:29.393 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:46:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:46:29.395 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:46:29.396 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:46:29.397 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:46:29.398 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:46:29.399 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:46:29.401 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:46:29.401 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:46:29.401 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:46:29.401 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:46:29.401 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:46:29.402 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:46:29.403 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:46:29.404 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:46:29.405 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:46:29.406 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:46:29.407 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:46:29.408 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:46:29.409 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:46:29.410 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:46:29.411 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:46:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:46:29.413 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:46:29.414 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:46:29.415 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:46:29.416 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:46:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:46:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:46:29.419 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:46:29.420 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:46:29.421 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:46:29.422 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:46:29.423 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:46:29.424 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:46:29.425 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:46:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:46:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:46:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:46:29.429 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:46:29.430 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:46:29.431 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:46:29.432 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:46:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:46:29.434 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:46:29.435 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:46:29.436 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:46:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:46:29.438 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:46:29.439 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:46:29.440 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:46:29.441 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:46:29.442 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:46:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:46:29.444 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:46:29.445 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:46:29.446 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:46:29.447 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:46:29.448 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:46:29.449 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:46:29.450 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:46:29.451 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:46:29.452 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:46:29.453 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:46:29.454 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:46:29.455 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:46:29.456 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:46:29.457 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:46:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:46:29.459 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:46:29.460 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:46:29.461 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:46:29.462 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:46:29.463 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:46:29.464 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:46:29.465 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:46:29.466 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:46:29.467 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:46:29.468 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:46:29.469 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:46:29.470 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:46:29.471 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:46:29.472 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:46:29.473 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:46:29.474 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:46:29.475 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:46:29.476 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:46:29.477 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:46:29.478 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:46:29.479 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:46:29.480 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:46:29.481 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:46:29.482 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:46:29.483 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:46:29.484 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:46:29.485 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:29.490 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.490 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:29.513 INFO:teuthology.orchestra.run.smithi027.stdout: fi 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T20:46:29.514 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:46:29.515 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:29.515 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.515 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-04-26T20:46:29.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-04-26T20:46:29.599 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-04-26T20:46:29.600 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:46:29.601 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:46:29.602 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:46:29.602 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:29.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:46:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-04-26T20:46:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:46:31.449 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-04-26T20:46:31.451 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-04-26T20:46:31.467 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.MjwfotYoMr 2024-04-26T20:46:31.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MjwfotYoMr 2024-04-26T20:46:31.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.MjwfotYoMr 2024-04-26T20:46:31.674 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.MjwfotYoMr is 6KB 2024-04-26T20:46:31.723 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.MjwfotYoMr 2024-04-26T20:46:31.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:46:31.805 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:46:31.806 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:46:31.806 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:46:31.806 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:46:31.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:31.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-04-26T20:46:39.777 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:46:39.777 DEBUG:teuthology.orchestra.run.smithi204:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:46:39.869 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:46:39.870 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:46:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:46:39.872 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:46:39.873 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:46:39.874 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:46:39.875 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:46:39.876 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:46:39.877 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:46:39.878 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:46:39.879 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:46:39.880 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:46:39.881 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:46:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:46:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:46:39.884 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:46:39.885 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:46:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:46:39.887 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:46:39.888 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:46:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:46:39.890 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:46:39.891 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:46:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:46:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:46:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:46:39.895 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:46:39.896 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:46:39.897 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:46:39.898 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:46:39.899 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:46:39.900 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:46:39.901 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:46:39.902 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:46:39.903 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:46:39.904 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:46:39.905 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:46:39.906 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:46:39.907 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:46:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:46:39.909 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:46:39.910 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:46:39.911 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:46:39.912 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:46:39.913 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:46:39.914 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:46:39.915 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:46:39.916 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:46:39.917 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:46:39.918 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:46:39.919 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:46:39.920 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:46:39.921 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:46:39.922 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:46:39.923 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:46:39.924 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:46:39.925 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:46:39.926 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:46:39.927 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:46:39.928 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:46:39.929 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:46:39.930 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:46:39.931 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:46:39.932 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:46:39.933 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:46:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:46:39.935 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:46:39.936 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:46:39.937 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:46:39.938 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:46:39.939 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:46:39.940 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:46:39.941 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:46:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:46:39.943 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:46:39.944 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:46:39.945 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:46:39.946 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:46:39.947 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:46:39.948 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:46:39.949 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:46:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:46:39.951 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:46:39.952 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:46:39.953 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:46:39.954 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:46:39.955 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:46:39.956 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:46:39.957 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:46:39.958 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:46:39.959 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:46:39.960 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:46:39.961 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:46:39.962 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:46:39.963 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:46:39.964 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:46:39.965 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:46:39.966 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:46:39.967 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:46:39.968 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:46:39.969 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:46:39.970 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:46:39.971 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:46:39.972 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:46:39.973 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:46:39.974 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:46:39.975 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:46:39.976 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:46:39.977 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:46:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:46:39.979 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:46:39.980 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:46:39.981 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:46:39.982 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:46:39.983 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:46:39.984 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:46:39.985 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:46:39.986 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:46:39.987 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:46:39.988 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:46:39.989 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:46:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:46:39.991 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:46:39.992 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:46:39.993 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:46:39.994 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:46:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:46:39.996 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:46:39.997 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:46:39.998 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:46:39.999 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:46:40.000 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:40.004 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:40.004 DEBUG:teuthology.orchestra.run.smithi204:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout: fi 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-04-26T20:46:40.026 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:40.027 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-04-26T20:46:40.028 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-04-26T20:46:40.109 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-04-26T20:46:40.110 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-04-26T20:46:40.110 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-04-26T20:46:40.110 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-04-26T20:46:40.110 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-04-26T20:46:40.110 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:46:40.111 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-04-26T20:46:40.112 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:46:40.113 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:46:40.115 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:46:40.115 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:40.115 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:46:40.671 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-04-26T20:46:42.041 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:46:42.071 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-04-26T20:46:42.073 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-26T20:46:42.108 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.uM9ih5Wd8B 2024-04-26T20:46:42.109 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uM9ih5Wd8B 2024-04-26T20:46:42.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.uM9ih5Wd8B 2024-04-26T20:46:42.328 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.uM9ih5Wd8B is 6KB 2024-04-26T20:46:42.377 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.uM9ih5Wd8B 2024-04-26T20:46:42.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:46:42.452 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:46:42.452 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:46:42.452 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:46:42.452 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:46:42.453 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:42.610 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-04-26T20:46:46.412 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:46:46.412 DEBUG:teuthology.orchestra.run.smithi094:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:46.485 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:46:46.486 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:46:46.487 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:46:46.488 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:46:46.489 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:46:46.490 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:46:46.491 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:46:46.492 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:46:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:46:46.494 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:46:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:46:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:46:46.497 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:46:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:46:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:46:46.500 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:46:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:46:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:46:46.503 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:46:46.504 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:46:46.505 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:46:46.506 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:46:46.507 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:46:46.508 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:46:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:46:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:46:46.511 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:46:46.512 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:46:46.513 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:46:46.514 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:46:46.515 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:46:46.516 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:46:46.517 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:46:46.518 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:46:46.519 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:46:46.520 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:46:46.521 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:46:46.522 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:46:46.523 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:46:46.524 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:46:46.525 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:46:46.526 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:46:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:46:46.528 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:46:46.529 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:46:46.530 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:46:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:46:46.532 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:46:46.533 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:46:46.534 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:46:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:46:46.536 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:46:46.537 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:46:46.538 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:46:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:46:46.540 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:46:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:46:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:46:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:46:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:46:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:46:46.546 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:46:46.547 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:46:46.548 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:46:46.549 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:46:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:46:46.551 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:46:46.552 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:46:46.553 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:46:46.554 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:46:46.555 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:46:46.556 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:46:46.557 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:46:46.558 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:46:46.559 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:46:46.560 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:46:46.561 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:46:46.562 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:46:46.563 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:46:46.564 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:46:46.565 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:46:46.566 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:46:46.567 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:46:46.568 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:46:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:46:46.570 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:46:46.571 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:46:46.572 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:46:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:46:46.574 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:46:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:46:46.576 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:46:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:46:46.578 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:46:46.579 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:46:46.580 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:46:46.581 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:46:46.582 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:46:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:46:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:46:46.585 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:46:46.586 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:46:46.587 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:46:46.588 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:46:46.589 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:46:46.590 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:46:46.591 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:46:46.592 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:46:46.593 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:46:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:46:46.595 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:46:46.596 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:46:46.597 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:46:46.598 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:46:46.599 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:46:46.600 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:46:46.601 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:46:46.602 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:46:46.603 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:46:46.604 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:46:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:46:46.606 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:46:46.607 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:46:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:46:46.609 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:46:46.610 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:46:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:46:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:46:46.613 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:46:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:46:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:46:46.616 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:46:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:46:46.618 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:46:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:46:46.620 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:46:46.621 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:46:46.622 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:46:46.623 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:46.627 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.627 DEBUG:teuthology.orchestra.run.smithi094:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: fi 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:46:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-26T20:46:46.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-04-26T20:46:46.722 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-04-26T20:46:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:46:46.724 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:46:46.725 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:46:46.725 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:46.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:46:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-04-26T20:46:48.582 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:46:48.616 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-04-26T20:46:48.619 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-26T20:46:48.650 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.XFirGWYG9C 2024-04-26T20:46:48.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XFirGWYG9C 2024-04-26T20:46:48.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.XFirGWYG9C 2024-04-26T20:46:48.858 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.XFirGWYG9C is 6KB 2024-04-26T20:46:48.907 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.XFirGWYG9C 2024-04-26T20:46:48.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:46:48.982 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:46:48.982 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:46:48.982 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:46:48.982 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:46:48.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:46:49.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-04-26T20:47:01.970 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T20:47:01.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:47:01.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:12.622 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-26T20:47:12.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:47:12.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:19.138 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T20:47:19.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T20:47:19.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:36.479 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T20:47:45.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:47:45.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:47.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-26T20:47:48.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T20:47:56.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:47:56.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:48:00.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:48:00.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:48:03.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-26T20:48:18.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:48:18.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:48:19.142 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:48:28.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T20:48:28.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:48:28.458 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-26T20:48:29.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T20:48:29.049 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:48:29.049 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-26T20:48:29.092 INFO:teuthology.orchestra.run.smithi094.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:48:29.092 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:48:29.092 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:48:29.092 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:48:30.093 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T20:48:30.094 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:48:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-04-26T20:48:30.281 DEBUG:teuthology.parallel:result is None 2024-04-26T20:48:56.237 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:49:08.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:49:08.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:49:08.659 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-26T20:49:09.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:49:09.279 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:49:09.279 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-26T20:49:09.331 INFO:teuthology.orchestra.run.smithi204.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:49:09.331 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:49:09.331 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:49:09.331 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:49:10.332 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-26T20:49:10.332 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:49:10.462 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-04-26T20:49:10.516 DEBUG:teuthology.parallel:result is None 2024-04-26T20:49:18.620 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:49:36.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:49:36.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:49:37.026 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-26T20:49:37.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:49:37.694 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:49:37.694 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-26T20:49:37.750 INFO:teuthology.orchestra.run.smithi027.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:49:37.750 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:49:37.750 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:49:37.750 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:49:38.751 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T20:49:38.751 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:49:38.882 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-04-26T20:49:38.942 DEBUG:teuthology.parallel:result is None 2024-04-26T20:49:38.942 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:49:38.950 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:49:38.950 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:49:38.956 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:49:38.966 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:49:39.028 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:49:39.035 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:49:39.061 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:49:39.061 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:49:39.069 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:49:39.076 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:49:39.149 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:49:39.156 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:49:39.156 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:49:39.161 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:49:39.168 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:49:39.254 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.256 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.264 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.287 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.288 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.296 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:49:39.301 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:49:39.309 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:49:39.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:49:39.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:49:39.322 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:49:39.403 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:49:39.412 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:49:39.412 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:49:39.419 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:49:39.426 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:49:39.478 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:39.562 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:39.669 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:39.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:39.852 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:49:39.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:49:39.937 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:40.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:40.140 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:40.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:40.313 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:49:40.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:49:40.390 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:40.476 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:49:40.560 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:40.640 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:49:40.721 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:49:40.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:49:40.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-04-26T20:49:40.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-26T20:49:40.817 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-04-26T20:49:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:49:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:49:40.978 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:49:41.365 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:49:41.373 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:49:41.373 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:49:41.422 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:49:41.454 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:49:41.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-04-26T20:49:41.607 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-04-26T20:49:41.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-26T20:49:41.760 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-26T20:49:41.764 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-04-26T20:49:41.922 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-04-26T20:49:41.928 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:49:41.929 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-04-26T20:49:41.982 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-04-26T20:49:41.982 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-26T20:49:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-26T20:49:42.036 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-04-26T20:49:42.083 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-04-26T20:49:42.084 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:49:42.084 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:49:42.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:49:42.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:49:42.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:49:42.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:49:42.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:49:42.324 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:49:42.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:49:42.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:49:42.477 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:49:42.549 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:49:42.558 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T20:49:42.563 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T20:49:42.680 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:49:42.696 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:49:42.697 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi094.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T20:55:00.035 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-04-26T20:55:00.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:55:00.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:55:00.150 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-26T20:55:00.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-26T20:55:00.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T20:55:00.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:55:00.412 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-26T20:55:00.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-26T20:55:00.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:55:00.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:55:00.667 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-26T20:55:00.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-26T20:55:00.829 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T20:55:00.840 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T20:55:00.841 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:55:00.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:55:00.849 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:55:00.849 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:55:00.857 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:55:00.857 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:55:00.944 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:55:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:55:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:55:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:55:00.986 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:55:00.993 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:55:01.059 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-26T20:55:01.063 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-04-26T20:55:01.069 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-04-26T20:55:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-26T20:55:01.095 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-04-26T20:55:01.103 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-04-26T20:55:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:55:01.135 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:55:01.147 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:55:01.175 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:55:01.175 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:55:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:55:01.310 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:55:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:55:01.319 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-26T20:55:01.320 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.320 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.320 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.320 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.334 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.345 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:55:01.347 INFO:teuthology.run_tasks:Running task install... 2024-04-26T20:55:01.355 DEBUG:teuthology.task.install:project ceph 2024-04-26T20:55:01.356 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T20:55:01.356 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T20:55:01.356 INFO:teuthology.task.install:Using flavor: default 2024-04-26T20:55:01.361 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T20:55:01.361 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T20:55:01.362 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:55:01.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:55:01.364 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:55:01.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:55:01.365 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:55:01.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:55:01.588 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:55:01.588 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:55:01.599 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:55:01.600 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:55:01.603 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:55:01.603 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:55:01.713 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:55:01.713 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:55:01.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:55:01.725 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:55:01.725 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:55:01.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:55:01.772 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:55:01.772 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:55:01.772 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:55:01.805 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:55:01.805 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:55:01.812 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:55:01.813 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:55:01.868 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:55:01.869 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:55:01.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:55:01.940 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:55:02.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:55:02.034 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:55:02.042 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:55:02.104 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-26T20:55:02.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-26T20:55:02.115 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-04-26T20:55:02.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-04-26T20:55:02.128 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:55:02.218 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-04-26T20:55:02.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-04-26T20:55:02.425 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-26T20:55:02.463 INFO:teuthology.orchestra.run.smithi204.stdout:48 files removed 2024-04-26T20:55:02.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:55:02.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:55:02.563 INFO:teuthology.orchestra.run.smithi027.stdout:48 files removed 2024-04-26T20:55:02.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:55:03.409 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 182 kB/s | 84 kB 00:00 2024-04-26T20:55:03.595 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 120 kB/s | 84 kB 00:00 2024-04-26T20:55:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 154 kB/s | 84 kB 00:00 2024-04-26T20:55:03.877 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-26T20:55:04.121 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-26T20:55:04.178 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-26T20:55:04.452 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-26T20:55:04.613 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-26T20:55:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T20:55:05.875 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T20:55:05.965 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-26T20:55:09.564 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-26T20:55:09.670 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-26T20:55:12.633 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:08 2024-04-26T20:55:14.662 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-26T20:55:14.714 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-26T20:55:14.876 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-26T20:55:16.413 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-26T20:55:16.432 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-26T20:55:18.386 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T20:55:18.615 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T20:55:20.019 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-26T20:55:21.636 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-26T20:55:23.826 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T20:55:24.509 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T20:55:24.614 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T20:55:26.945 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:26.946 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:26.967 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:55:26.968 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:55:26.969 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:55:26.971 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:55:26.973 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:55:26.977 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:55:26.978 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:55:27.048 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-26T20:55:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T20:55:27.071 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-26T20:55:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T20:55:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-26T20:55:27.071 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:55:27.072 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:55:27.073 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-26T20:55:27.074 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:55:27.075 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:55:27.076 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:55:27.077 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:55:27.078 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:55:27.079 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:55:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:55:27.081 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:55:27.082 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:55:27.083 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-26T20:55:27.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:55:27.085 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 222 M 2024-04-26T20:55:27.085 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-26T20:55:27.117 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:27.119 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:27.140 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:55:27.141 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:55:27.143 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:55:27.144 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:55:27.146 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:55:27.150 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:55:27.151 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:55:27.229 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:55:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:55:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:55:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:55:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:55:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:55:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:55:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:55:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:55:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:55:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:55:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 4 Packages 2024-04-26T20:55:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:55:27.264 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 222 M 2024-04-26T20:55:27.264 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-26T20:55:28.011 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-26T20:55:28.203 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.5 MB/s | 889 kB 00:00 2024-04-26T20:55:28.262 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.5 MB/s | 145 kB 00:00 2024-04-26T20:55:28.357 INFO:teuthology.orchestra.run.smithi027.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-26T20:55:28.512 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 9.5 MB/s | 2.4 MB 00:00 2024-04-26T20:55:28.696 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-04-26T20:55:29.121 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-04-26T20:55:29.149 INFO:teuthology.orchestra.run.smithi027.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 889 kB 00:00 2024-04-26T20:55:29.230 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-04-26T20:55:29.308 INFO:teuthology.orchestra.run.smithi027.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 921 kB/s | 145 kB 00:00 2024-04-26T20:55:29.771 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-26T20:55:30.664 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 7.3 MB/s | 21 MB 00:02 2024-04-26T20:55:30.683 INFO:teuthology.orchestra.run.smithi027.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-26T20:55:30.872 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 6.5 MB/s | 11 MB 00:01 2024-04-26T20:55:30.897 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 108 kB/s | 25 kB 00:00 2024-04-26T20:55:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-26T20:55:31.106 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 298 kB/s | 27 kB 00:00 2024-04-26T20:55:31.207 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 320 kB/s | 32 kB 00:00 2024-04-26T20:55:31.292 INFO:teuthology.orchestra.run.smithi027.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-26T20:55:31.390 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.0 MB/s | 747 kB 00:00 2024-04-26T20:55:31.392 INFO:teuthology.orchestra.run.smithi027.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-04-26T20:55:31.524 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.2 MB/s | 163 kB 00:00 2024-04-26T20:55:31.649 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.0 MB/s | 127 kB 00:00 2024-04-26T20:55:31.910 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 6.0 MB/s | 17 MB 00:02 2024-04-26T20:55:31.969 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 1.5 MB/s | 501 kB 00:00 2024-04-26T20:55:32.077 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 419 kB/s | 45 kB 00:00 2024-04-26T20:55:32.173 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:32.175 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:55:32.194 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:55:32.196 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:55:32.197 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:55:32.198 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:55:32.200 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:55:32.204 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:55:32.205 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:55:32.228 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 907 kB/s | 135 kB 00:00 2024-04-26T20:55:32.274 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-04-26T20:55:32.290 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T20:55:32.290 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:55:32.291 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:55:32.292 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:55:32.294 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:55:32.295 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:55:32.296 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:55:32.297 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:55:32.298 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:55:32.299 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:55:32.300 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:55:32.301 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 4 Packages 2024-04-26T20:55:32.302 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:55:32.303 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 222 M 2024-04-26T20:55:32.303 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-04-26T20:55:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.3 MB/s | 161 kB 00:00 2024-04-26T20:55:32.359 INFO:teuthology.orchestra.run.smithi027.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-04-26T20:55:32.962 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 528 kB/s | 321 kB 00:00 2024-04-26T20:55:33.079 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 298 kB 00:00 2024-04-26T20:55:33.221 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 706 kB/s | 99 kB 00:00 2024-04-26T20:55:33.346 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.7 MB/s | 5.4 MB 00:01 2024-04-26T20:55:33.373 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 570 kB/s | 85 kB 00:00 2024-04-26T20:55:33.496 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-26T20:55:33.614 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 294 kB/s | 34 kB 00:00 2024-04-26T20:55:33.739 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.1 MB/s | 142 kB 00:00 2024-04-26T20:55:33.909 INFO:teuthology.orchestra.run.smithi027.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 3.6 MB/s | 21 MB 00:05 2024-04-26T20:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-26T20:55:34.001 INFO:teuthology.orchestra.run.smithi027.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 278 kB/s | 25 kB 00:00 2024-04-26T20:55:34.114 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.0 MB/s | 3.1 MB 00:00 2024-04-26T20:55:34.415 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.2 MB/s | 3.5 MB 00:00 2024-04-26T20:55:34.732 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 788 kB/s | 249 kB 00:00 2024-04-26T20:55:34.785 INFO:teuthology.orchestra.run.smithi027.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.4 MB/s | 11 MB 00:02 2024-04-26T20:55:34.789 INFO:teuthology.orchestra.run.smithi204.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-26T20:55:34.840 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 459 kB/s | 49 kB 00:00 2024-04-26T20:55:34.877 INFO:teuthology.orchestra.run.smithi027.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 225 kB 00:00 2024-04-26T20:55:34.923 INFO:teuthology.orchestra.run.smithi204.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.1 MB/s | 145 kB 00:00 2024-04-26T20:55:34.957 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 142 kB/s | 16 kB 00:00 2024-04-26T20:55:34.977 INFO:teuthology.orchestra.run.smithi027.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 273 kB/s | 27 kB 00:00 2024-04-26T20:55:35.086 INFO:teuthology.orchestra.run.smithi027.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 295 kB/s | 32 kB 00:00 2024-04-26T20:55:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 273 kB 00:00 2024-04-26T20:55:35.369 INFO:teuthology.orchestra.run.smithi027.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.6 MB/s | 747 kB 00:00 2024-04-26T20:55:35.461 INFO:teuthology.orchestra.run.smithi027.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 163 kB 00:00 2024-04-26T20:55:35.578 INFO:teuthology.orchestra.run.smithi027.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.1 MB/s | 127 kB 00:00 2024-04-26T20:55:35.708 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-26T20:55:35.775 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-26T20:55:35.792 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 221 kB/s | 18 kB 00:00 2024-04-26T20:55:35.820 INFO:teuthology.orchestra.run.smithi027.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 2.0 MB/s | 501 kB 00:00 2024-04-26T20:55:36.426 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.8 MB/s | 49 MB 00:05 2024-04-26T20:55:36.451 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 119 kB/s | 80 kB 00:00 2024-04-26T20:55:36.490 INFO:teuthology.orchestra.run.smithi204.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-26T20:55:36.601 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-26T20:55:36.660 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 838 kB/s | 195 kB 00:00 2024-04-26T20:55:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 892 kB/s | 803 kB 00:00 2024-04-26T20:55:36.727 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T20:55:36.761 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T20:55:36.803 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-26T20:55:36.853 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-26T20:55:36.887 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-26T20:55:36.948 INFO:teuthology.orchestra.run.smithi204.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-26T20:55:36.954 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-04-26T20:55:36.979 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-26T20:55:37.038 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-04-26T20:55:37.063 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-26T20:55:37.097 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-26T20:55:37.122 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-04-26T20:55:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 275 kB/s | 18 kB 00:00 2024-04-26T20:55:37.190 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-04-26T20:55:37.223 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T20:55:37.257 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 682 kB/s | 45 kB 00:00 2024-04-26T20:55:37.349 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-26T20:55:37.370 INFO:teuthology.orchestra.run.smithi027.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.5 MB/s | 5.4 MB 00:01 2024-04-26T20:55:37.425 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-26T20:55:37.458 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-04-26T20:55:37.462 INFO:teuthology.orchestra.run.smithi027.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 500 kB/s | 45 kB 00:00 2024-04-26T20:55:37.483 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 827 kB/s | 47 kB 00:00 2024-04-26T20:55:37.525 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T20:55:37.537 INFO:teuthology.orchestra.run.smithi027.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T20:55:37.600 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-26T20:55:37.604 INFO:teuthology.orchestra.run.smithi027.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T20:55:37.626 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-04-26T20:55:37.651 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 282 kB/s | 35 kB 00:00 2024-04-26T20:55:37.721 INFO:teuthology.orchestra.run.smithi027.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 321 kB 00:00 2024-04-26T20:55:37.847 INFO:teuthology.orchestra.run.smithi027.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T20:55:37.877 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-04-26T20:55:37.922 INFO:teuthology.orchestra.run.smithi027.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T20:55:37.999 INFO:teuthology.orchestra.run.smithi204.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-26T20:55:38.005 INFO:teuthology.orchestra.run.smithi027.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T20:55:38.035 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-04-26T20:55:38.102 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T20:55:38.107 INFO:teuthology.orchestra.run.smithi204.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-04-26T20:55:38.245 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-04-26T20:55:38.353 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.3 MB/s | 4.7 MB 00:00 2024-04-26T20:55:38.420 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T20:55:38.470 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-26T20:55:38.546 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T20:55:38.612 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-26T20:55:38.671 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 443 kB/s | 26 kB 00:00 2024-04-26T20:55:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-04-26T20:55:38.731 INFO:teuthology.orchestra.run.smithi027.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-26T20:55:38.855 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 7.7 MB/s | 5.8 MB 00:00 2024-04-26T20:55:38.889 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-26T20:55:38.914 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 601 kB/s | 35 kB 00:00 2024-04-26T20:55:38.982 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-26T20:55:38.989 INFO:teuthology.orchestra.run.smithi027.stdout:(27/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 2.2 MB/s | 17 MB 00:07 2024-04-26T20:55:39.023 INFO:teuthology.orchestra.run.smithi027.stdout:(28/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 587 kB/s | 171 kB 00:00 2024-04-26T20:55:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T20:55:39.056 INFO:teuthology.orchestra.run.smithi027.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 521 kB/s | 34 kB 00:00 2024-04-26T20:55:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T20:55:39.199 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 7.7 MB/s | 6.0 MB 00:00 2024-04-26T20:55:39.224 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 442 kB/s | 77 kB 00:00 2024-04-26T20:55:39.257 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 729 kB/s | 42 kB 00:00 2024-04-26T20:55:39.291 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-26T20:55:39.325 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-26T20:55:39.367 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-26T20:55:39.392 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 768 kB/s | 51 kB 00:00 2024-04-26T20:55:39.426 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 928 kB/s | 54 kB 00:00 2024-04-26T20:55:39.560 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-04-26T20:55:39.627 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-04-26T20:55:39.652 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 483 kB/s | 44 kB 00:00 2024-04-26T20:55:39.694 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-26T20:55:39.719 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 541 kB/s | 36 kB 00:00 2024-04-26T20:55:39.753 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 819 kB/s | 47 kB 00:00 2024-04-26T20:55:39.787 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T20:55:39.812 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-26T20:55:39.846 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T20:55:39.871 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T20:55:39.905 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-26T20:55:40.057 INFO:teuthology.orchestra.run.smithi027.stdout:(31/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.1 MB/s | 49 MB 00:06 2024-04-26T20:55:40.122 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-04-26T20:55:40.157 INFO:teuthology.orchestra.run.smithi027.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.5 MB/s | 249 kB 00:00 2024-04-26T20:55:40.190 INFO:teuthology.orchestra.run.smithi204.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 5.1 MB/s | 11 MB 00:02 2024-04-26T20:55:40.241 INFO:teuthology.orchestra.run.smithi027.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 602 kB/s | 49 kB 00:00 2024-04-26T20:55:40.257 INFO:teuthology.orchestra.run.smithi204.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 385 kB/s | 25 kB 00:00 2024-04-26T20:55:40.308 INFO:teuthology.orchestra.run.smithi027.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 251 kB/s | 16 kB 00:00 2024-04-26T20:55:40.356 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-26T20:55:40.413 INFO:teuthology.orchestra.run.smithi027.stdout:(35/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.6 MB/s | 3.5 MB 00:01 2024-04-26T20:55:40.467 INFO:teuthology.orchestra.run.smithi027.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 273 kB 00:00 2024-04-26T20:55:40.542 INFO:teuthology.orchestra.run.smithi027.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 247 kB/s | 18 kB 00:00 2024-04-26T20:55:40.651 INFO:teuthology.orchestra.run.smithi027.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-26T20:55:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 605 kB/s | 80 kB 00:00 2024-04-26T20:55:40.752 INFO:teuthology.orchestra.run.smithi027.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-26T20:55:40.810 INFO:teuthology.orchestra.run.smithi027.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-26T20:55:40.856 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-26T20:55:40.869 INFO:teuthology.orchestra.run.smithi027.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-26T20:55:40.898 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 707 kB/s | 548 kB 00:00 2024-04-26T20:55:41.003 INFO:teuthology.orchestra.run.smithi027.stdout:(43/152): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-04-26T20:55:41.036 INFO:teuthology.orchestra.run.smithi027.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 641 kB/s | 106 kB 00:00 2024-04-26T20:55:41.040 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 453 kB/s | 309 kB 00:00 2024-04-26T20:55:41.111 INFO:teuthology.orchestra.run.smithi027.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-04-26T20:55:41.115 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 365 kB/s | 26 kB 00:00 2024-04-26T20:55:41.168 INFO:teuthology.orchestra.run.smithi027.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 951 kB/s | 125 kB 00:00 2024-04-26T20:55:41.203 INFO:teuthology.orchestra.run.smithi027.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T20:55:41.229 INFO:teuthology.orchestra.run.smithi027.stdout:(48/152): python3-pycparser-2.20-6.el9.noarch.r 601 kB/s | 135 kB 00:00 2024-04-26T20:55:41.232 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-26T20:55:41.257 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): grpc-data-1.46.7-10.el9.noarch.rpm 49 kB/s | 19 kB 00:00 2024-04-26T20:55:41.283 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2024-04-26T20:55:41.309 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 893 kB/s | 67 kB 00:00 2024-04-26T20:55:41.396 INFO:teuthology.orchestra.run.smithi027.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 644 kB/s | 107 kB 00:00 2024-04-26T20:55:41.401 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-26T20:55:41.446 INFO:teuthology.orchestra.run.smithi027.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-26T20:55:41.459 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 752 kB/s | 151 kB 00:00 2024-04-26T20:55:41.471 INFO:teuthology.orchestra.run.smithi027.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-04-26T20:55:41.485 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 524 kB/s | 43 kB 00:00 2024-04-26T20:55:41.527 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-04-26T20:55:41.547 INFO:teuthology.orchestra.run.smithi027.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-26T20:55:41.552 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-04-26T20:55:41.578 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-26T20:55:41.580 INFO:teuthology.orchestra.run.smithi027.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-26T20:55:41.605 INFO:teuthology.orchestra.run.smithi027.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-04-26T20:55:41.611 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-26T20:55:41.645 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-26T20:55:41.655 INFO:teuthology.orchestra.run.smithi027.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 229 kB/s | 11 kB 00:00 2024-04-26T20:55:41.672 INFO:teuthology.orchestra.run.smithi027.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 277 kB/s | 18 kB 00:00 2024-04-26T20:55:41.687 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-26T20:55:41.714 INFO:teuthology.orchestra.run.smithi027.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T20:55:41.729 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T20:55:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 782 kB/s | 45 kB 00:00 2024-04-26T20:55:41.784 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-04-26T20:55:41.821 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T20:55:41.823 INFO:teuthology.orchestra.run.smithi027.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-26T20:55:41.863 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 138 kB/s | 11 kB 00:00 2024-04-26T20:55:41.897 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-04-26T20:55:41.909 INFO:teuthology.orchestra.run.smithi027.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 300 kB/s | 25 kB 00:00 2024-04-26T20:55:41.949 INFO:teuthology.orchestra.run.smithi027.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-26T20:55:41.972 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-04-26T20:55:41.974 INFO:teuthology.orchestra.run.smithi027.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 742 kB/s | 47 kB 00:00 2024-04-26T20:55:41.997 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-26T20:55:42.022 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-04-26T20:55:42.048 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-04-26T20:55:42.049 INFO:teuthology.orchestra.run.smithi027.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-04-26T20:55:42.083 INFO:teuthology.orchestra.run.smithi027.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-26T20:55:42.098 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 625 kB/s | 46 kB 00:00 2024-04-26T20:55:42.108 INFO:teuthology.orchestra.run.smithi027.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 35 kB 00:00 2024-04-26T20:55:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T20:55:42.150 INFO:teuthology.orchestra.run.smithi027.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 292 kB 00:00 2024-04-26T20:55:42.182 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-26T20:55:42.210 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2024-04-26T20:55:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T20:55:42.258 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-04-26T20:55:42.282 INFO:teuthology.orchestra.run.smithi204.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.4 MB/s | 21 MB 00:08 2024-04-26T20:55:42.292 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T20:55:42.309 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 425 kB/s | 28 kB 00:00 2024-04-26T20:55:42.326 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 462 kB/s | 31 kB 00:00 2024-04-26T20:55:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T20:55:42.410 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-26T20:55:42.424 INFO:teuthology.orchestra.run.smithi204.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 225 kB 00:00 2024-04-26T20:55:42.435 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-04-26T20:55:42.461 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-26T20:55:42.475 INFO:teuthology.orchestra.run.smithi027.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-26T20:55:42.494 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T20:55:42.508 INFO:teuthology.orchestra.run.smithi204.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 328 kB/s | 27 kB 00:00 2024-04-26T20:55:42.519 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-26T20:55:42.570 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-04-26T20:55:42.592 INFO:teuthology.orchestra.run.smithi204.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 385 kB/s | 32 kB 00:00 2024-04-26T20:55:42.604 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-26T20:55:42.637 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-26T20:55:42.659 INFO:teuthology.orchestra.run.smithi027.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-04-26T20:55:42.713 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T20:55:42.759 INFO:teuthology.orchestra.run.smithi027.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.0 MB/s | 4.7 MB 00:00 2024-04-26T20:55:42.835 INFO:teuthology.orchestra.run.smithi027.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-26T20:55:42.850 INFO:teuthology.orchestra.run.smithi204.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.8 MB/s | 747 kB 00:00 2024-04-26T20:55:42.860 INFO:teuthology.orchestra.run.smithi027.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 667 kB/s | 133 kB 00:00 2024-04-26T20:55:42.919 INFO:teuthology.orchestra.run.smithi027.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T20:55:42.934 INFO:teuthology.orchestra.run.smithi204.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T20:55:43.009 INFO:teuthology.orchestra.run.smithi204.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T20:55:43.011 INFO:teuthology.orchestra.run.smithi027.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-26T20:55:43.030 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-26T20:55:43.078 INFO:teuthology.orchestra.run.smithi027.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.7 MB/s | 244 kB 00:00 2024-04-26T20:55:43.138 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.4 MB/s | 3.4 MB 00:00 2024-04-26T20:55:43.144 INFO:teuthology.orchestra.run.smithi027.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-26T20:55:43.159 INFO:teuthology.orchestra.run.smithi204.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 3.3 MB/s | 501 kB 00:00 2024-04-26T20:55:43.211 INFO:teuthology.orchestra.run.smithi027.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-04-26T20:55:43.270 INFO:teuthology.orchestra.run.smithi027.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 334 kB/s | 19 kB 00:00 2024-04-26T20:55:43.305 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.7 MB/s | 3.1 MB 00:00 2024-04-26T20:55:43.362 INFO:teuthology.orchestra.run.smithi027.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T20:55:43.363 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-04-26T20:55:43.380 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:55:43.381 INFO:teuthology.orchestra.run.smithi094.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-26T20:55:43.421 INFO:teuthology.orchestra.run.smithi027.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 611 kB/s | 35 kB 00:00 2024-04-26T20:55:43.476 INFO:teuthology.orchestra.run.smithi204.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-26T20:55:43.560 INFO:teuthology.orchestra.run.smithi204.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 549 kB/s | 45 kB 00:00 2024-04-26T20:55:43.621 INFO:teuthology.orchestra.run.smithi027.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 6.7 MB/s | 5.8 MB 00:00 2024-04-26T20:55:43.677 INFO:teuthology.orchestra.run.smithi204.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.1 MB/s | 135 kB 00:00 2024-04-26T20:55:43.711 INFO:teuthology.orchestra.run.smithi027.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-26T20:55:43.780 INFO:teuthology.orchestra.run.smithi027.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-26T20:55:43.786 INFO:teuthology.orchestra.run.smithi204.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T20:55:43.839 INFO:teuthology.orchestra.run.smithi027.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T20:55:43.897 INFO:teuthology.orchestra.run.smithi027.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 735 kB/s | 42 kB 00:00 2024-04-26T20:55:43.902 INFO:teuthology.orchestra.run.smithi204.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 321 kB 00:00 2024-04-26T20:55:44.011 INFO:teuthology.orchestra.run.smithi204.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 298 kB 00:00 2024-04-26T20:55:44.014 INFO:teuthology.orchestra.run.smithi027.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-04-26T20:55:44.072 INFO:teuthology.orchestra.run.smithi027.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-26T20:55:44.078 INFO:teuthology.orchestra.run.smithi204.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T20:55:44.106 INFO:teuthology.orchestra.run.smithi027.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T20:55:44.145 INFO:teuthology.orchestra.run.smithi204.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T20:55:44.215 INFO:teuthology.orchestra.run.smithi027.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-04-26T20:55:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T20:55:44.273 INFO:teuthology.orchestra.run.smithi027.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 307 kB/s | 51 kB 00:00 2024-04-26T20:55:44.299 INFO:teuthology.orchestra.run.smithi027.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 647 kB/s | 54 kB 00:00 2024-04-26T20:55:44.333 INFO:teuthology.orchestra.run.smithi027.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 718 kB/s | 42 kB 00:00 2024-04-26T20:55:44.358 INFO:teuthology.orchestra.run.smithi027.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 763 kB/s | 44 kB 00:00 2024-04-26T20:55:44.400 INFO:teuthology.orchestra.run.smithi027.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-26T20:55:44.442 INFO:teuthology.orchestra.run.smithi027.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 433 kB/s | 36 kB 00:00 2024-04-26T20:55:44.458 INFO:teuthology.orchestra.run.smithi027.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 823 kB/s | 47 kB 00:00 2024-04-26T20:55:44.509 INFO:teuthology.orchestra.run.smithi027.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T20:55:44.529 INFO:teuthology.orchestra.run.smithi204.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-04-26T20:55:44.542 INFO:teuthology.orchestra.run.smithi027.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-26T20:55:44.568 INFO:teuthology.orchestra.run.smithi027.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T20:55:44.602 INFO:teuthology.orchestra.run.smithi027.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T20:55:44.626 INFO:teuthology.orchestra.run.smithi204.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-26T20:55:44.669 INFO:teuthology.orchestra.run.smithi027.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-04-26T20:55:44.710 INFO:teuthology.orchestra.run.smithi027.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-04-26T20:55:44.713 INFO:teuthology.orchestra.run.smithi204.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 399 kB/s | 34 kB 00:00 2024-04-26T20:55:44.788 INFO:teuthology.orchestra.run.smithi204.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-26T20:55:44.819 INFO:teuthology.orchestra.run.smithi027.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.7 MB/s | 863 kB 00:00 2024-04-26T20:55:44.972 INFO:teuthology.orchestra.run.smithi204.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.7 MB/s | 3.1 MB 00:00 2024-04-26T20:55:45.531 INFO:teuthology.orchestra.run.smithi204.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 4.8 MB/s | 3.5 MB 00:00 2024-04-26T20:55:45.553 INFO:teuthology.orchestra.run.smithi027.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 652 kB/s | 548 kB 00:00 2024-04-26T20:55:45.586 INFO:teuthology.orchestra.run.smithi027.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 445 kB/s | 309 kB 00:00 2024-04-26T20:55:45.645 INFO:teuthology.orchestra.run.smithi027.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 214 kB/s | 19 kB 00:00 2024-04-26T20:55:45.745 INFO:teuthology.orchestra.run.smithi027.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-04-26T20:55:45.845 INFO:teuthology.orchestra.run.smithi027.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 491 kB/s | 49 kB 00:00 2024-04-26T20:55:45.946 INFO:teuthology.orchestra.run.smithi027.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 680 kB/s | 67 kB 00:00 2024-04-26T20:55:46.072 INFO:teuthology.orchestra.run.smithi204.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.4 MB/s | 49 MB 00:05 2024-04-26T20:55:46.106 INFO:teuthology.orchestra.run.smithi204.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 433 kB/s | 249 kB 00:00 2024-04-26T20:55:46.148 INFO:teuthology.orchestra.run.smithi204.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 660 kB/s | 49 kB 00:00 2024-04-26T20:55:46.171 INFO:teuthology.orchestra.run.smithi027.stdout:(110/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.5 MB/s | 16 MB 00:01 2024-04-26T20:55:46.190 INFO:teuthology.orchestra.run.smithi204.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 198 kB/s | 16 kB 00:00 2024-04-26T20:55:46.205 INFO:teuthology.orchestra.run.smithi027.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 585 kB/s | 151 kB 00:00 2024-04-26T20:55:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-26T20:55:46.223 INFO:teuthology.orchestra.run.smithi204.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.6 MB/s | 273 kB 00:00 2024-04-26T20:55:46.290 INFO:teuthology.orchestra.run.smithi204.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 276 kB/s | 18 kB 00:00 2024-04-26T20:55:46.314 INFO:teuthology.orchestra.run.smithi027.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-26T20:55:46.423 INFO:teuthology.orchestra.run.smithi027.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-26T20:55:46.440 INFO:teuthology.orchestra.run.smithi204.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.0 MB/s | 762 kB 00:00 2024-04-26T20:55:46.463 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-26T20:55:46.463 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-26T20:55:46.474 INFO:teuthology.orchestra.run.smithi204.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 440 kB/s | 80 kB 00:00 2024-04-26T20:55:46.489 INFO:teuthology.orchestra.run.smithi027.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 420 kB/s | 43 kB 00:00 2024-04-26T20:55:46.523 INFO:teuthology.orchestra.run.smithi027.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-26T20:55:46.574 INFO:teuthology.orchestra.run.smithi204.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T20:55:46.615 INFO:teuthology.orchestra.run.smithi027.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-26T20:55:46.648 INFO:teuthology.orchestra.run.smithi027.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-26T20:55:46.666 INFO:teuthology.orchestra.run.smithi204.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T20:55:46.724 INFO:teuthology.orchestra.run.smithi027.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T20:55:46.782 INFO:teuthology.orchestra.run.smithi027.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-26T20:55:46.824 INFO:teuthology.orchestra.run.smithi027.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 356 kB/s | 35 kB 00:00 2024-04-26T20:55:46.833 INFO:teuthology.orchestra.run.smithi204.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-26T20:55:46.884 INFO:teuthology.orchestra.run.smithi204.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-26T20:55:46.900 INFO:teuthology.orchestra.run.smithi027.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-26T20:55:46.951 INFO:teuthology.orchestra.run.smithi204.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T20:55:47.017 INFO:teuthology.orchestra.run.smithi027.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T20:55:47.026 INFO:teuthology.orchestra.run.smithi204.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-04-26T20:55:47.050 INFO:teuthology.orchestra.run.smithi027.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 969 kB/s | 144 kB 00:00 2024-04-26T20:55:47.051 INFO:teuthology.orchestra.run.smithi204.stdout:(46/152): python3-ply-3.11-14.el9.noarch.rpm 492 kB/s | 106 kB 00:00 2024-04-26T20:55:47.076 INFO:teuthology.orchestra.run.smithi204.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-26T20:55:47.092 INFO:teuthology.orchestra.run.smithi027.stdout:(124/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 912 kB/s | 839 kB 00:00 2024-04-26T20:55:47.110 INFO:teuthology.orchestra.run.smithi204.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T20:55:47.117 INFO:teuthology.orchestra.run.smithi027.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T20:55:47.151 INFO:teuthology.orchestra.run.smithi027.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-26T20:55:47.193 INFO:teuthology.orchestra.run.smithi027.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-26T20:55:47.218 INFO:teuthology.orchestra.run.smithi027.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-26T20:55:47.252 INFO:teuthology.orchestra.run.smithi027.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-26T20:55:47.361 INFO:teuthology.orchestra.run.smithi027.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T20:55:47.386 INFO:teuthology.orchestra.run.smithi027.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 278 kB/s | 46 kB 00:00 2024-04-26T20:55:47.403 INFO:teuthology.orchestra.run.smithi027.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 526 kB/s | 79 kB 00:00 2024-04-26T20:55:47.470 INFO:teuthology.orchestra.run.smithi027.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2024-04-26T20:55:47.520 INFO:teuthology.orchestra.run.smithi027.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-26T20:55:47.545 INFO:teuthology.orchestra.run.smithi027.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-26T20:55:47.579 INFO:teuthology.orchestra.run.smithi027.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 833 kB/s | 90 kB 00:00 2024-04-26T20:55:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 284 kB/s | 28 kB 00:00 2024-04-26T20:55:47.671 INFO:teuthology.orchestra.run.smithi027.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 247 kB/s | 31 kB 00:00 2024-04-26T20:55:47.730 INFO:teuthology.orchestra.run.smithi027.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-26T20:55:47.747 INFO:teuthology.orchestra.run.smithi027.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-04-26T20:55:47.764 INFO:teuthology.orchestra.run.smithi027.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-04-26T20:55:47.822 INFO:teuthology.orchestra.run.smithi027.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 559 kB/s | 51 kB 00:00 2024-04-26T20:55:47.864 INFO:teuthology.orchestra.run.smithi027.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-26T20:55:47.889 INFO:teuthology.orchestra.run.smithi027.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 717 kB/s | 90 kB 00:00 2024-04-26T20:55:47.894 INFO:teuthology.orchestra.run.smithi204.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 136 kB/s | 107 kB 00:00 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi027.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-26T20:55:47.965 INFO:teuthology.orchestra.run.smithi027.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-26T20:55:47.998 INFO:teuthology.orchestra.run.smithi027.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T20:55:48.099 INFO:teuthology.orchestra.run.smithi027.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-26T20:55:48.308 INFO:teuthology.orchestra.run.smithi027.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-26T20:55:48.458 INFO:teuthology.orchestra.run.smithi027.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-26T20:55:48.486 INFO:teuthology.orchestra.run.smithi204.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-04-26T20:55:48.594 INFO:teuthology.orchestra.run.smithi204.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 447 kB/s | 689 kB 00:01 2024-04-26T20:55:48.728 INFO:teuthology.orchestra.run.smithi204.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 571 kB/s | 943 kB 00:01 2024-04-26T20:55:48.841 INFO:teuthology.orchestra.run.smithi027.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-26T20:55:49.066 INFO:teuthology.orchestra.run.smithi027.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.1 MB/s | 3.4 MB 00:01 2024-04-26T20:55:49.078 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:55:49.079 INFO:teuthology.orchestra.run.smithi027.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-26T20:55:49.163 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-26T20:55:49.164 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-26T20:55:49.370 INFO:teuthology.orchestra.run.smithi204.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 42 kB/s | 33 kB 00:00 2024-04-26T20:55:49.512 INFO:teuthology.orchestra.run.smithi204.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 306 kB/s | 313 kB 00:01 2024-04-26T20:55:49.996 INFO:teuthology.orchestra.run.smithi204.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 29 kB/s | 18 kB 00:00 2024-04-26T20:55:50.054 INFO:teuthology.orchestra.run.smithi204.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-04-26T20:55:50.713 INFO:teuthology.orchestra.run.smithi204.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-04-26T20:55:50.805 INFO:teuthology.orchestra.run.smithi204.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 198 kB/s | 160 kB 00:00 2024-04-26T20:55:52.013 INFO:teuthology.orchestra.run.smithi204.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 483 kB/s | 582 kB 00:01 2024-04-26T20:55:52.080 INFO:teuthology.orchestra.run.smithi204.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 485 kB/s | 662 kB 00:01 2024-04-26T20:55:52.124 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-26T20:55:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-26T20:55:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-26T20:55:52.597 INFO:teuthology.orchestra.run.smithi204.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-04-26T20:55:52.705 INFO:teuthology.orchestra.run.smithi204.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-04-26T20:55:53.148 INFO:teuthology.orchestra.run.smithi204.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 693 kB/s | 3.0 MB 00:04 2024-04-26T20:55:53.748 INFO:teuthology.orchestra.run.smithi204.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 254 kB/s | 292 kB 00:01 2024-04-26T20:55:53.781 INFO:teuthology.orchestra.run.smithi204.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 175 kB/s | 188 kB 00:01 2024-04-26T20:55:53.848 INFO:teuthology.orchestra.run.smithi204.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-04-26T20:55:55.046 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-26T20:55:55.046 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-26T20:55:55.440 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-26T20:55:55.555 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:55:55.703 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:55:56.053 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:55:56.128 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:55:56.166 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:55:56.316 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:55:56.382 INFO:teuthology.orchestra.run.smithi204.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 642 kB/s | 1.6 MB 00:02 2024-04-26T20:55:56.749 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:55:56.863 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:55:57.030 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:55:57.064 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:55:57.188 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:55:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:55:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:55:57.622 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:55:57.670 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:55:57.842 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:55:57.870 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:55:57.966 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:55:57.997 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:55:58.007 INFO:teuthology.orchestra.run.smithi204.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 561 kB/s | 911 kB 00:01 2024-04-26T20:55:58.063 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:55:58.377 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:55:58.448 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:55:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:55:58.782 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:55:58.807 INFO:teuthology.orchestra.run.smithi204.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 167 kB/s | 133 kB 00:00 2024-04-26T20:55:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:55:58.871 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:55:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:55:59.027 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:55:59.091 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:55:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:55:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:55:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:55:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:55:59.434 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:55:59.487 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:55:59.595 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:55:59.637 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:55:59.649 INFO:teuthology.orchestra.run.smithi204.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 772 kB/s | 4.4 MB 00:05 2024-04-26T20:55:59.693 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:55:59.728 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:55:59.831 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:55:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:55:59.949 INFO:teuthology.orchestra.run.smithi204.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 783 kB/s | 4.7 MB 00:06 2024-04-26T20:55:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:56:00.140 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:56:00.206 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:56:00.708 INFO:teuthology.orchestra.run.smithi204.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 128 kB/s | 87 kB 00:00 2024-04-26T20:56:00.834 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:56:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:56:01.002 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:56:01.049 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:56:01.158 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:56:01.419 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:56:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:56:01.560 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:56:01.591 INFO:teuthology.orchestra.run.smithi204.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 532 kB/s | 1.0 MB 00:01 2024-04-26T20:56:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:56:01.713 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:56:01.753 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:56:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:56:01.872 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-26T20:56:01.878 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:56:01.943 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:56:01.993 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:56:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:56:02.084 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:56:02.190 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:56:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:56:02.241 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:56:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:56:02.276 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:56:02.297 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:56:02.346 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:56:02.428 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:56:02.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:56:02.467 INFO:teuthology.orchestra.run.smithi204.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 280 kB/s | 244 kB 00:00 2024-04-26T20:56:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:56:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:56:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:56:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:56:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:56:02.856 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:56:02.966 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:56:02.971 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:56:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:56:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:56:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:56:03.167 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:56:03.182 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:56:03.201 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:56:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:56:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:56:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:56:03.332 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:56:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:56:03.405 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:56:03.417 INFO:teuthology.orchestra.run.smithi204.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 261 kB/s | 248 kB 00:00 2024-04-26T20:56:03.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:56:03.502 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:56:03.673 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:56:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:56:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:56:03.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:56:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:56:03.856 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:56:03.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:56:04.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:56:04.017 INFO:teuthology.orchestra.run.smithi204.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-26T20:56:04.040 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:56:04.073 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:56:04.074 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:56:04.180 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:56:04.207 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:56:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:56:04.593 INFO:teuthology.orchestra.run.smithi204.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 33 kB/s | 19 kB 00:00 2024-04-26T20:56:04.627 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:56:04.684 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:56:04.886 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:56:05.016 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:56:05.022 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:56:05.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:56:05.082 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:56:05.108 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:56:05.115 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:56:05.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:56:05.247 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:56:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:56:05.326 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:56:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:56:05.384 INFO:teuthology.orchestra.run.smithi204.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 218 kB/s | 172 kB 00:00 2024-04-26T20:56:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:56:05.426 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:56:05.432 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:56:05.467 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:56:05.556 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:56:05.629 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:56:05.667 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:56:05.699 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:56:05.726 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:56:05.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:56:05.836 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:56:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:56:05.879 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:56:05.937 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:56:05.976 INFO:teuthology.orchestra.run.smithi204.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 59 kB/s | 35 kB 00:00 2024-04-26T20:56:05.977 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:56:06.103 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:56:06.168 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:56:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:56:06.352 INFO:teuthology.orchestra.run.smithi204.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 785 kB/s | 5.8 MB 00:07 2024-04-26T20:56:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:56:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:56:06.680 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:56:06.829 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:56:07.193 INFO:teuthology.orchestra.run.smithi204.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 299 kB/s | 250 kB 00:00 2024-04-26T20:56:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:56:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:56:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:56:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:56:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:56:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:56:07.826 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:56:07.895 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:56:07.979 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:56:08.043 INFO:teuthology.orchestra.run.smithi204.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 262 kB/s | 222 kB 00:00 2024-04-26T20:56:08.079 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:56:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:56:08.177 INFO:teuthology.orchestra.run.smithi204.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 818 kB/s | 6.0 MB 00:07 2024-04-26T20:56:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:56:08.255 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:56:08.301 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:56:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:56:08.414 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:56:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:56:08.525 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:56:08.568 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:56:08.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:56:08.667 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:56:08.715 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:56:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:56:08.808 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:56:08.815 INFO:teuthology.orchestra.run.smithi204.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 100 kB/s | 77 kB 00:00 2024-04-26T20:56:08.853 INFO:teuthology.orchestra.run.smithi204.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-04-26T20:56:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:56:09.001 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:56:09.046 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:09.127 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:56:09.166 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:09.202 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:56:09.209 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:56:09.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:56:09.266 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:56:09.286 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:56:09.464 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:56:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:56:09.669 INFO:teuthology.orchestra.run.smithi204.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 195 kB/s | 159 kB 00:00 2024-04-26T20:56:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:56:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:56:09.720 INFO:teuthology.orchestra.run.smithi204.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 298 kB/s | 269 kB 00:00 2024-04-26T20:56:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:56:09.817 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:56:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:56:09.857 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:09.894 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:56:09.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:56:09.958 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:09.993 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:56:10.000 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:56:10.050 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:56:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:56:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:56:10.319 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:56:10.353 INFO:teuthology.orchestra.run.smithi204.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 81 kB/s | 51 kB 00:00 2024-04-26T20:56:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:56:10.428 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:56:10.500 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:56:10.554 INFO:teuthology.orchestra.run.smithi204.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 316 kB/s | 279 kB 00:00 2024-04-26T20:56:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:56:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:56:11.012 INFO:teuthology.orchestra.run.smithi204.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-04-26T20:56:11.638 INFO:teuthology.orchestra.run.smithi204.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-04-26T20:56:11.717 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:56:11.774 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:56:11.810 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:56:11.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:56:11.995 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:56:12.104 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:56:12.150 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:56:12.280 INFO:teuthology.orchestra.run.smithi204.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-04-26T20:56:12.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:56:12.422 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:56:12.480 INFO:teuthology.orchestra.run.smithi204.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 803 kB/s | 5.1 MB 00:06 2024-04-26T20:56:12.563 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:56:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:13.147 INFO:teuthology.orchestra.run.smithi204.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 54 kB/s | 36 kB 00:00 2024-04-26T20:56:13.247 INFO:teuthology.orchestra.run.smithi204.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 228 kB/s | 221 kB 00:00 2024-04-26T20:56:13.456 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:56:13.602 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:56:13.653 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:56:13.835 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:56:13.835 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:56:13.835 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:56:13.836 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:13.839 INFO:teuthology.orchestra.run.smithi204.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-04-26T20:56:13.879 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:56:14.215 INFO:teuthology.orchestra.run.smithi204.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 315 kB/s | 305 kB 00:00 2024-04-26T20:56:14.506 INFO:teuthology.orchestra.run.smithi204.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 87 kB/s | 58 kB 00:00 2024-04-26T20:56:14.957 INFO:teuthology.orchestra.run.smithi204.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 127 kB/s | 94 kB 00:00 2024-04-26T20:56:15.151 INFO:teuthology.orchestra.run.smithi204.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 99 kB/s | 64 kB 00:00 2024-04-26T20:56:15.284 INFO:teuthology.orchestra.run.smithi204.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 169 kB/s | 22 kB 00:00 2024-04-26T20:56:15.290 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:56:15.413 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:56:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:56:15.510 INFO:teuthology.orchestra.run.smithi204.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.8 MB/s | 863 kB 00:00 2024-04-26T20:56:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:56:15.576 INFO:teuthology.orchestra.run.smithi204.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-04-26T20:56:15.885 INFO:teuthology.orchestra.run.smithi204.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-26T20:56:15.918 INFO:teuthology.orchestra.run.smithi204.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-26T20:56:15.928 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:56:15.966 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:15.986 INFO:teuthology.orchestra.run.smithi204.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-26T20:56:16.094 INFO:teuthology.orchestra.run.smithi204.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-26T20:56:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:16.156 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:56:16.187 INFO:teuthology.orchestra.run.smithi204.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 532 kB/s | 49 kB 00:00 2024-04-26T20:56:16.216 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:56:16.286 INFO:teuthology.orchestra.run.smithi204.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 686 kB/s | 67 kB 00:00 2024-04-26T20:56:16.403 INFO:teuthology.orchestra.run.smithi204.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-26T20:56:16.437 INFO:teuthology.orchestra.run.smithi204.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T20:56:16.579 INFO:teuthology.orchestra.run.smithi204.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-26T20:56:16.613 INFO:teuthology.orchestra.run.smithi204.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-26T20:56:16.655 INFO:teuthology.orchestra.run.smithi204.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 581 kB/s | 43 kB 00:00 2024-04-26T20:56:16.730 INFO:teuthology.orchestra.run.smithi204.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-26T20:56:16.780 INFO:teuthology.orchestra.run.smithi204.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-26T20:56:16.808 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:56:16.839 INFO:teuthology.orchestra.run.smithi204.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-26T20:56:16.844 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:16.906 INFO:teuthology.orchestra.run.smithi204.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-26T20:56:16.946 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:16.971 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:56:16.989 INFO:teuthology.orchestra.run.smithi204.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-26T20:56:17.015 INFO:teuthology.orchestra.run.smithi204.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-04-26T20:56:17.123 INFO:teuthology.orchestra.run.smithi204.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-26T20:56:17.215 INFO:teuthology.orchestra.run.smithi204.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-26T20:56:17.249 INFO:teuthology.orchestra.run.smithi204.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-26T20:56:17.299 INFO:teuthology.orchestra.run.smithi204.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-26T20:56:17.324 INFO:teuthology.orchestra.run.smithi204.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-26T20:56:17.383 INFO:teuthology.orchestra.run.smithi204.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 279 kB/s | 23 kB 00:00 2024-04-26T20:56:17.409 INFO:teuthology.orchestra.run.smithi204.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-26T20:56:17.468 INFO:teuthology.orchestra.run.smithi204.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-26T20:56:17.567 INFO:teuthology.orchestra.run.smithi204.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-26T20:56:17.593 INFO:teuthology.orchestra.run.smithi204.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-04-26T20:56:17.651 INFO:teuthology.orchestra.run.smithi204.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 949 kB/s | 79 kB 00:00 2024-04-26T20:56:17.710 INFO:teuthology.orchestra.run.smithi204.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 495 kB/s | 58 kB 00:00 2024-04-26T20:56:17.744 INFO:teuthology.orchestra.run.smithi204.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T20:56:17.794 INFO:teuthology.orchestra.run.smithi204.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 199 kB/s | 16 kB 00:00 2024-04-26T20:56:17.844 INFO:teuthology.orchestra.run.smithi204.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 899 kB/s | 90 kB 00:00 2024-04-26T20:56:17.878 INFO:teuthology.orchestra.run.smithi204.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-04-26T20:56:17.953 INFO:teuthology.orchestra.run.smithi204.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-04-26T20:56:17.987 INFO:teuthology.orchestra.run.smithi204.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T20:56:18.071 INFO:teuthology.orchestra.run.smithi204.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-04-26T20:56:18.096 INFO:teuthology.orchestra.run.smithi204.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 417 kB/s | 59 kB 00:00 2024-04-26T20:56:18.171 INFO:teuthology.orchestra.run.smithi204.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-04-26T20:56:18.222 INFO:teuthology.orchestra.run.smithi204.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-26T20:56:18.272 INFO:teuthology.orchestra.run.smithi204.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 899 kB/s | 90 kB 00:00 2024-04-26T20:56:18.313 INFO:teuthology.orchestra.run.smithi204.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-26T20:56:18.356 INFO:teuthology.orchestra.run.smithi204.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-04-26T20:56:18.523 INFO:teuthology.orchestra.run.smithi204.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-26T20:56:18.573 INFO:teuthology.orchestra.run.smithi204.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 741 kB/s | 192 kB 00:00 2024-04-26T20:56:18.940 INFO:teuthology.orchestra.run.smithi204.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 8.2 MB/s | 3.4 MB 00:00 2024-04-26T20:56:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-26T20:56:19.382 INFO:teuthology.orchestra.run.smithi204.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-26T20:56:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:19.899 INFO:teuthology.orchestra.run.smithi204.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 2.4 MB 00:00 2024-04-26T20:56:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:56:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:56:20.736 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:56:20.736 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:56:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-26T20:56:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-26T20:56:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-26T20:56:27.193 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:27.420 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:56:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:56:27.467 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:27.467 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:56:27.467 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:29.088 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:56:29.164 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:56:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:56:29.482 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:29.482 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:56:29.482 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:56:29.483 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:56:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:56:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:56:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:56:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:56:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:56:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:56:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:56:30.306 INFO:teuthology.orchestra.run.smithi204.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 846 kB/s | 16 MB 00:19 2024-04-26T20:56:30.314 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:56:30.314 INFO:teuthology.orchestra.run.smithi204.stdout:Total 3.8 MB/s | 222 MB 00:58 2024-04-26T20:56:30.334 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:56:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:56:30.627 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:56:30.712 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:56:30.812 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:56:30.979 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:56:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:56:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:56:31.118 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:56:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:56:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:56:31.864 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:56:31.950 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:56:32.017 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:56:33.155 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-04-26T20:56:33.331 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:56:33.369 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:56:33.398 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-04-26T20:56:33.398 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-04-26T20:56:34.465 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:56:34.494 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-04-26T20:56:34.562 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:34.641 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:56:34.804 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:56:34.818 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:56:34.855 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:56:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:56:34.858 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:34.858 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:56:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:56:34.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:56:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:56:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:56:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:56:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:56:36.194 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-04-26T20:56:36.194 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-04-26T20:56:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:56:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:56:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:56:36.432 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:36.432 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:56:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:56:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:56:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:56:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:56:36.953 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:56:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:56:36.973 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:36.973 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:56:36.973 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:56:36.973 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:56:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:56:37.026 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:37.523 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:56:37.595 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:56:37.595 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:37.595 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:56:37.596 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:56:37.596 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:56:37.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:37.742 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:56:37.811 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:56:37.915 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:56:38.202 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:56:38.280 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:56:38.375 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:56:38.491 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:56:38.666 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:56:38.723 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:56:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:56:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:56:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:56:39.547 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:56:39.609 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:56:39.728 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:56:39.765 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:56:41.139 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:56:41.183 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:56:42.024 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:56:42.077 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:56:42.154 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:56:42.191 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:56:42.232 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:56:42.352 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:56:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:56:42.441 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-04-26T20:56:42.537 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:56:42.562 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:56:42.599 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:56:42.622 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:56:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:56:42.649 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:56:42.747 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:56:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:56:42.811 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:56:42.819 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:56:42.849 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:56:42.917 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:42.966 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:56:42.978 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:56:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:56:43.136 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:56:43.207 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:56:43.413 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:56:43.529 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:43.529 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:56:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:43.713 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:56:43.748 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:56:43.871 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:56:43.914 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:56:44.239 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:56:44.284 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:56:44.303 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:56:44.313 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:56:44.326 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:56:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:56:44.475 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:56:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:56:44.525 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:44.526 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:56:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:44.532 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:56:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:56:44.623 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:56:44.654 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:56:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:56:44.686 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:44.687 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:56:44.687 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:56:44.687 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:56:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:44.721 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:56:44.931 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:56:45.065 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:56:45.140 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:45.158 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:56:45.167 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:56:45.214 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:56:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:56:45.420 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:56:45.457 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:56:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:56:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:56:45.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:56:45.596 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:56:45.633 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:56:45.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:56:45.680 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:56:45.680 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:56:45.769 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:56:45.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:56:45.828 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:56:45.891 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:56:45.939 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:56:45.959 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:56:45.978 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:56:46.095 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:56:46.164 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:56:46.205 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:56:46.226 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:56:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:56:46.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:46.260 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:56:46.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:56:46.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:56:46.458 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:56:46.490 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:56:46.605 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:56:46.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:56:46.765 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:56:46.965 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:56:47.030 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:56:47.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:56:47.776 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:56:47.813 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:56:47.858 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:56:47.899 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:56:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:56:48.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:56:48.301 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:56:48.380 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:56:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:56:48.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:56:48.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:56:48.612 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:56:48.655 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:56:48.717 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:56:48.775 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:56:48.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:56:49.028 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:56:49.066 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:56:49.122 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:56:49.260 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:56:49.307 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:56:49.356 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:56:49.389 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:56:49.422 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:56:49.608 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:56:49.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:56:49.846 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:56:49.886 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:56:49.933 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:56:50.000 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:56:50.059 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:56:50.109 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:56:50.154 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:56:50.203 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:56:50.278 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:56:50.327 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:56:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:56:50.469 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:56:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:56:50.530 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:56:50.575 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:56:50.624 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:56:50.643 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:56:50.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:56:50.718 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:56:50.790 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:56:50.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:56:51.074 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:56:51.135 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:56:51.163 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:56:51.805 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:56:51.853 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:56:51.853 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:56:51.853 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:56:51.853 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:56:51.854 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:56:51.855 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:56:51.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:56:51.867 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:56:51.867 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:56:51.867 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:56:51.867 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:56:51.868 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:56:51.869 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:56:51.870 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:56:51.871 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:56:51.872 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:56:51.873 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:56:51.874 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:56:51.875 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:56:51.876 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:56:51.876 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:56:51.888 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:56:52.005 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:56:52.074 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:56:52.130 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:52.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:56:52.178 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:52.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:56:52.239 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:56:52.448 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:56:52.487 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:56:52.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:56:53.439 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:56:53.576 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:56:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:56:53.918 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:56:54.021 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:56:54.215 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:56:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:56:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:56:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:56:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:56:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:56:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:56:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:56:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:56:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:56:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:56:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T20:56:54.328 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-26T20:56:54.410 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:56:54.456 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:56:54.457 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:56:54.589 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:56:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:56:54.999 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:55.044 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:56:55.044 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:56:55.044 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:56:55.206 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:56:55.319 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:56:55.363 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:56:55.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:56:55.789 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:56:55.840 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:55.971 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:56:56.005 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:56:56.085 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:56:56.621 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:56:56.660 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:56.760 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:56:56.787 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:56:59.522 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:56:59.558 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:57:00.215 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:57:00.262 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:57:00.445 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:57:00.446 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:57:00.446 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:57:00.446 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:00.496 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:57:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:57:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:57:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:57:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:57:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:57:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:57:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:57:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:57:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:57:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:57:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:57:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:57:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:57:03.587 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.588 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:57:03.589 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.590 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:03.591 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:57:03.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:57:03.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:57:03.596 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:57:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-26T20:57:04.196 DEBUG:teuthology.parallel:result is None 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-04-26T20:57:13.547 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:13.763 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:57:13.810 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:57:13.810 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:13.810 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:57:13.810 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:15.439 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:57:15.520 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:57:15.520 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:15.520 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:57:15.520 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:57:15.521 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:57:15.521 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:15.773 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:57:15.844 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:16.324 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:57:16.390 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:16.532 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:57:16.595 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:57:16.672 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:57:16.911 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:57:16.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:57:17.055 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:57:17.180 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:57:17.347 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:57:17.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:57:17.658 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:57:17.695 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:57:18.319 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:57:18.355 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:57:18.409 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:57:18.519 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:57:18.549 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:57:19.838 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:57:19.885 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:57:20.981 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:57:21.009 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:57:21.155 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:57:21.331 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:57:21.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:57:21.433 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:57:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:57:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:57:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:57:21.434 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:21.487 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:57:21.526 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:57:21.637 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:57:21.708 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:57:22.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:57:22.757 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:57:22.829 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:57:22.829 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:22.829 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:57:22.830 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:57:22.830 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:57:22.830 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:22.891 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:57:22.960 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:57:22.960 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:22.960 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:57:22.960 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:23.042 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:57:23.113 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:23.467 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:57:23.537 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:28.557 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:57:28.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:57:28.701 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:57:28.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:57:28.794 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:57:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:57:29.170 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:57:29.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:57:30.095 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:57:30.146 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:57:30.203 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:57:31.562 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:57:31.670 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:57:31.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:57:31.922 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:57:31.968 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:57:31.968 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:57:32.094 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:57:32.142 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:57:32.142 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:57:32.316 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:57:32.456 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:57:32.770 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:57:32.771 INFO:teuthology.orchestra.run.smithi204.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:57:32.771 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:38.524 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:57:38.525 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:57:38.526 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:57:38.527 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:57:38.528 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:57:38.530 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:57:38.531 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:57:38.532 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:57:38.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:57:38.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:57:38.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:57:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:57:38.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:57:38.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:57:38.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:57:41.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.036 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.037 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:57:41.038 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:57:41.039 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:57:41.040 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:57:41.041 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:57:41.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:57:41.043 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:57:41.044 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:57:41.046 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:57:41.047 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:57:41.048 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:57:41.048 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T20:57:41.048 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-04-26T20:57:41.635 DEBUG:teuthology.parallel:result is None 2024-04-26T20:57:41.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:41.856 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:57:41.936 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:41.937 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:41.937 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:57:41.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:42.159 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:57:42.234 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:42.234 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:42.234 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:57:42.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:42.465 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:57:42.522 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:42.522 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:57:42.522 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:57:42.524 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T20:57:42.524 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:57:42.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:57:42.624 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:57:42.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:57:42.713 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:57:42.713 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:57:42.788 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T20:57:42.788 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:57:42.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:57:42.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:57:43.014 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:57:43.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:57:43.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:57:43.223 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:57:43.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:57:43.292 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:57:43.400 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T20:57:43.400 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:57:43.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:57:43.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:57:43.635 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:57:43.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:57:43.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:57:43.823 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:57:43.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:57:43.889 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:57:43.994 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T20:57:43.994 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:57:43.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:57:44.086 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:57:44.178 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T20:57:44.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:57:44.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:57:44.367 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T20:57:44.367 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:57:44.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:57:44.537 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T20:57:44.639 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T20:57:44.639 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:44.640 INFO:tasks.cephadm:Cluster fsid is a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:57:44.640 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T20:57:44.640 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.94', 'mon.c': '172.21.15.204'} 2024-04-26T20:57:44.640 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-04-26T20:57:44.640 INFO:tasks.cephadm:First mgr is x 2024-04-26T20:57:44.640 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T20:57:44.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-04-26T20:57:44.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-26T20:57:44.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-04-26T20:57:44.888 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T20:57:44.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:45.028 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T20:57:45.250 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:57:45.252 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T20:57:45.252 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:45.775 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 20:57 /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:45.776 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:46.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 20:57 /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:46.442 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:47.095 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 20:57 /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:47.096 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:47.159 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:47.220 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T20:57:47.278 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T20:57:47.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T20:57:47.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T20:57:47.295 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T20:57:49.245 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T20:57:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T20:57:50.094 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T20:58:28.322 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T20:58:28.323 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-04-26T20:58:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T20:58:32.722 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T20:58:32.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:58:32.723 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T20:58:32.723 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-26T20:58:32.723 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T20:58:32.723 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-26T20:58:32.723 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T20:58:32.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-04-26T20:58:32.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-26T20:58:32.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-04-26T20:58:33.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-04-26T20:58:33.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-26T20:58:33.194 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-04-26T20:58:33.267 INFO:tasks.cephadm:Writing seed config... 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T20:58:33.268 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T20:58:33.269 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T20:58:33.270 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T20:58:33.271 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-26T20:58:33.272 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T20:58:33.272 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T20:58:33.461 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a125c556-040f-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T20:58:33.462 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T20:58:33.470 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T20:58:33.518 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T20:58:33.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid a125c556-040f-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:58:33.793 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:58:33.794 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'a125c556-040f-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-04-26T20:58:33.794 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T20:58:33.794 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-04-26T20:58:33.853 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T20:58:33.853 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-04-26T20:58:33.854 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-04-26T20:58:33.869 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T20:58:33.869 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T20:58:33.884 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T20:58:33.884 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T20:58:33.906 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-04-26T20:58:33.924 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-04-26T20:58:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-04-26T20:58:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-04-26T20:58:33.983 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T20:58:33.983 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T20:58:33.983 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-04-26T20:58:33.983 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-04-26T20:58:33.998 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T20:58:33.998 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T20:58:34.011 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T20:58:34.011 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T20:58:34.026 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-04-26T20:58:34.040 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-04-26T20:58:34.040 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-04-26T20:58:34.040 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-04-26T20:58:34.040 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:34.041 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 139808354454688 on /run/cephadm/a125c556-040f-11ef-bc93-c7b262605968.lock 2024-04-26T20:58:34.041 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 139808354454688 acquired on /run/cephadm/a125c556-040f-11ef-bc93-c7b262605968.lock 2024-04-26T20:58:34.041 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-04-26T20:58:34.041 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-04-26T20:58:34.042 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-04-26T20:58:34.050 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-04-26T20:58:34.050 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-04-26T20:58:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T20:58:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T20:58:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-26T20:58:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T20:58:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-04-26T20:58:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-04-26T20:58:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T20:58:34.065 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T20:58:34.066 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T20:58:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T20:58:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T20:58:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T20:58:41.479 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T20:58:42.697 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-04-26T20:58:42.697 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-04-26T20:58:43.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCDFSxm1p2DDBAAqwgx4lc4Sw2s9NFCGLEkmg== 2024-04-26T20:58:45.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCEFSxmYmJCFxAAHEkPudC+ObMcC9VW5kT60Q== 2024-04-26T20:58:46.335 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQCFFSxmqpB6JhAAQD0/XT/ET6anm+Mi+n8MKA== 2024-04-26T20:58:46.336 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-04-26T20:58:47.574 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T20:58:47.574 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T20:58:47.574 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:47.574 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T20:58:47.575 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-04-26T20:58:48.777 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-04-26T20:58:49.212 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T20:58:49.475 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T20:58:49.790 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a125c556-040f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968.target. 2024-04-26T20:58:49.790 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a125c556-040f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968.target. 2024-04-26T20:58:50.188 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a 2024-04-26T20:58:50.188 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T20:58:50.494 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968.target.wants/ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968@.service. 2024-04-26T20:58:51.191 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:50 smithi027 podman[27694]: 2024-04-26 20:58:50.934253076 +0000 UTC m=+0.150668782 container create 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:58:51.191 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 podman[27694]: 2024-04-26 20:58:51.188460525 +0000 UTC m=+0.404876265 container init 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T20:58:51.318 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T20:58:51.318 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T20:58:51.334 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T20:58:51.334 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T20:58:51.335 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T20:58:51.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T20:58:51.335 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-04-26T20:58:51.335 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-04-26T20:58:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 podman[27694]: 2024-04-26 20:58:51.200648985 +0000 UTC m=+0.417064699 container start 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T20:58:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 ceph-mon[27713]: mkfs a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 bash[27694]: 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 2024-04-26T20:58:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 ceph-mon[27713]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T20:58:51.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:51 smithi027 systemd[1]: Started Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: monmap epoch 1 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: last_changed 2024-04-26T20:58:46.841103+0000 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T20:58:52.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: min_mon_release 19 (squid) 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: election_strategy: 1 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: fsmap 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: mgrmap e1: no daemons active 2024-04-26T20:58:52.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:52 smithi027 ceph-mon[27713]: from='client.? 172.21.15.27:0/1539561191' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-04-26T20:58:52.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.795315s) 2024-04-26T20:58:52.756 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T20:58:52.756 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T20:58:52.756 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:52.756 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.981+0000 7fd8d1a00700 1 Processor -- start 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.981+0000 7fd8d1a00700 1 -- start start 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.981+0000 7fd8d1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.981+0000 7fd8d1a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc112670 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.982+0000 7fd8caa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.983+0000 7fd8caa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37048/0 (socket says 172.21.15.27:37048) 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.983+0000 7fd8caa00700 1 -- 172.21.15.27:0/1606602912 learned_addr learned my addr 172.21.15.27:0/1606602912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.983+0000 7fd8caa00700 1 -- 172.21.15.27:0/1606602912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1127f0 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.984+0000 7fd8caa00700 1 --2- 172.21.15.27:0/1606602912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0098a0 tx=0x7fd8bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9076589ed1915ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.984+0000 7fd8c9600700 1 -- 172.21.15.27:0/1606602912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc004160 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.984+0000 7fd8c9600700 1 -- 172.21.15.27:0/1606602912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8bc004300 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.984+0000 7fd8c9600700 1 -- 172.21.15.27:0/1606602912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc0036a0 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1606602912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 msgr2=0x7fd8cc1120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 --2- 172.21.15.27:0/1606602912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0098a0 tx=0x7fd8bc009bb0 comp rx=0 tx=0).stop 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1606602912 shutdown_connections 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 --2- 172.21.15.27:0/1606602912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1120a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1606602912 >> 172.21.15.27:0/1606602912 conn(0x7fd8cc07e3d0 msgr2=0x7fd8cc0807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1606602912 shutdown_connections 2024-04-26T20:58:52.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.985+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1606602912 wait complete. 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.986+0000 7fd8d1a00700 1 Processor -- start 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.986+0000 7fd8d1a00700 1 -- start start 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.986+0000 7fd8d1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.986+0000 7fd8d1a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc112d20 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.987+0000 7fd8caa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.987+0000 7fd8caa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37056/0 (socket says 172.21.15.27:37056) 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.987+0000 7fd8caa00700 1 -- 172.21.15.27:0/1539561191 learned_addr learned my addr 172.21.15.27:0/1539561191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.987+0000 7fd8caa00700 1 -- 172.21.15.27:0/1539561191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1a3170 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8caa00700 1 --2- 172.21.15.27:0/1539561191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc003db0 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8bc011630 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc011950 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc1a3400 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.988+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc1a60a0 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.989+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8bc01a460 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.989+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd8bc01b780 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.989+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc064a80 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:51.993+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd8bc01b980 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.073+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd8cc02e5e0 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.074+0000 7fd8c3400700 1 -- 172.21.15.27:0/1539561191 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd8bc01bb60 con 0x7fd8cc10fcb0 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 msgr2=0x7fd8cc1a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 --2- 172.21.15.27:0/1539561191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc004750 comp rx=0 tx=0).stop 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 shutdown_connections 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 --2- 172.21.15.27:0/1539561191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8cc10fcb0 0x7fd8cc1a2c30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 >> 172.21.15.27:0/1539561191 conn(0x7fd8cc07e3d0 msgr2=0x7fd8cc0806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 shutdown_connections 2024-04-26T20:58:52.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:52.075+0000 7fd8d1a00700 1 -- 172.21.15.27:0/1539561191 wait complete. 2024-04-26T20:58:52.765 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-04-26T20:58:52.765 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T20:58:53.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:53 smithi027 ceph-mon[27713]: from='client.? 172.21.15.27:0/3677855940' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T20:58:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:53 smithi027 ceph-mon[27713]: from='client.? 172.21.15.27:0/3677855940' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T20:58:54.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:54.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T20:58:54.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:54.195 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T20:58:54.196 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-04-26T20:58:54.196 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T20:58:54.196 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T20:58:54.197 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T20:58:54.197 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T20:58:54.197 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:54.197 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T20:58:54.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T20:58:54.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:58:54.198 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T20:58:54.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T20:58:54.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T20:58:54.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T20:58:54.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T20:58:54.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T20:58:54.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T20:58:54.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0c3200700 1 Processor -- start 2024-04-26T20:58:54.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0c3200700 1 -- start start 2024-04-26T20:58:54.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0c3200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0c3200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112670 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0bbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0bbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37062/0 (socket says 172.21.15.27:37062) 2024-04-26T20:58:54.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0bbe00700 1 -- 172.21.15.27:0/663022506 learned_addr learned my addr 172.21.15.27:0/663022506 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:54.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.396+0000 7fa0bbe00700 1 -- 172.21.15.27:0/663022506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1127f0 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.397+0000 7fa0bbe00700 1 --2- 172.21.15.27:0/663022506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0098a0 tx=0x7fa0ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2ed2b725783d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.397+0000 7fa0baa00700 1 -- 172.21.15.27:0/663022506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac004160 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.397+0000 7fa0baa00700 1 -- 172.21.15.27:0/663022506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac004300 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.397+0000 7fa0baa00700 1 -- 172.21.15.27:0/663022506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac0036a0 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 -- 172.21.15.27:0/663022506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 msgr2=0x7fa0bc1120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 --2- 172.21.15.27:0/663022506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0098a0 tx=0x7fa0ac009bb0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 -- 172.21.15.27:0/663022506 shutdown_connections 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 --2- 172.21.15.27:0/663022506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1120a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 -- 172.21.15.27:0/663022506 >> 172.21.15.27:0/663022506 conn(0x7fa0bc07e3d0 msgr2=0x7fa0bc0807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 -- 172.21.15.27:0/663022506 shutdown_connections 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.398+0000 7fa0c3200700 1 -- 172.21.15.27:0/663022506 wait complete. 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.399+0000 7fa0c3200700 1 Processor -- start 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.399+0000 7fa0c3200700 1 -- start start 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0c3200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0c3200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112d20 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0bbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0bbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37066/0 (socket says 172.21.15.27:37066) 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0bbe00700 1 -- 172.21.15.27:0/3677855940 learned_addr learned my addr 172.21.15.27:0/3677855940 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0bbe00700 1 -- 172.21.15.27:0/3677855940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1a3170 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.400+0000 7fa0bbe00700 1 --2- 172.21.15.27:0/3677855940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000c00 tx=0x7fa0ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac003db0 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac011630 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac011950 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc1a3400 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1a60a0 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa0ac01a460 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.401+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa0ac01b780 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.402+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0bc064a80 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.405+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa0ac01ba20 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.483+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa0bc02e620 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.495+0000 7fa0b8c00700 1 -- 172.21.15.27:0/3677855940 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fa0ac01bc00 con 0x7fa0bc10fcb0 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.496+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 msgr2=0x7fa0bc1a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.496+0000 7fa0c3200700 1 --2- 172.21.15.27:0/3677855940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000c00 tx=0x7fa0ac004750 comp rx=0 tx=0).stop 2024-04-26T20:58:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.496+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 shutdown_connections 2024-04-26T20:58:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.496+0000 7fa0c3200700 1 --2- 172.21.15.27:0/3677855940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc10fcb0 0x7fa0bc1a2c30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.497+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 >> 172.21.15.27:0/3677855940 conn(0x7fa0bc07e3d0 msgr2=0x7fa0bc0806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.497+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 shutdown_connections 2024-04-26T20:58:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:53.497+0000 7fa0c3200700 1 -- 172.21.15.27:0/3677855940 wait complete. 2024-04-26T20:58:54.211 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-04-26T20:58:55.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:55 smithi027 ceph-mon[27713]: from='client.? 172.21.15.27:0/815030268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.865+0000 7fadffc00700 1 Processor -- start 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.865+0000 7fadffc00700 1 -- start start 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.866+0000 7fadffc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.866+0000 7fadffc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810b9e0 con 0x7fadf810aff0 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.866+0000 7fadfd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.866+0000 7fadfd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37074/0 (socket says 172.21.15.27:37074) 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.866+0000 7fadfd200700 1 -- 172.21.15.27:0/2811459033 learned_addr learned my addr 172.21.15.27:0/2811459033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.867+0000 7fadfd200700 1 -- 172.21.15.27:0/2811459033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf810c1f0 con 0x7fadf810aff0 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.867+0000 7fadfd200700 1 --2- 172.21.15.27:0/2811459033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27478064f191baf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.868+0000 7fadf7400700 1 -- 172.21.15.27:0/2811459033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec004160 con 0x7fadf810aff0 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.868+0000 7fadf7400700 1 -- 172.21.15.27:0/2811459033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fadec004300 con 0x7fadf810aff0 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.868+0000 7fadf7400700 1 -- 172.21.15.27:0/2811459033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec0119e0 con 0x7fadf810aff0 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.868+0000 7fadffc00700 1 -- 172.21.15.27:0/2811459033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 msgr2=0x7fadf810b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.868+0000 7fadffc00700 1 --2- 172.21.15.27:0/2811459033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec009bb0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.869+0000 7fadffc00700 1 -- 172.21.15.27:0/2811459033 shutdown_connections 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.869+0000 7fadffc00700 1 --2- 172.21.15.27:0/2811459033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf810b410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.869+0000 7fadffc00700 1 -- 172.21.15.27:0/2811459033 >> 172.21.15.27:0/2811459033 conn(0x7fadf807e3d0 msgr2=0x7fadf807e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.869+0000 7fadffc00700 1 -- 172.21.15.27:0/2811459033 shutdown_connections 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.869+0000 7fadffc00700 1 -- 172.21.15.27:0/2811459033 wait complete. 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.870+0000 7fadffc00700 1 Processor -- start 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.870+0000 7fadffc00700 1 -- start start 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadffc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadffc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810f420 con 0x7fadf810aff0 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadfd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadfd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37082/0 (socket says 172.21.15.27:37082) 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadfd200700 1 -- 172.21.15.27:0/815030268 learned_addr learned my addr 172.21.15.27:0/815030268 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.871+0000 7fadfd200700 1 -- 172.21.15.27:0/815030268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf819f3e0 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadfd200700 1 --2- 172.21.15.27:0/815030268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadec006b20 tx=0x7fadec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec003a60 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fadec003c00 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf819f670 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec024780 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a2360 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fadec022020 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.872+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fadec02dde0 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.874+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf8064a80 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.877+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fadec01aba0 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.955+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fadf81a28a0 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.955+0000 7fadf5600700 1 -- 172.21.15.27:0/815030268 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fadec02d3f0 con 0x7fadf810aff0 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.956+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 msgr2=0x7fadf819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.956+0000 7fadffc00700 1 --2- 172.21.15.27:0/815030268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadec006b20 tx=0x7fadec011a60 comp rx=0 tx=0).stop 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.956+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 shutdown_connections 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.956+0000 7fadffc00700 1 --2- 172.21.15.27:0/815030268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadf810aff0 0x7fadf819eea0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.956+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 >> 172.21.15.27:0/815030268 conn(0x7fadf807e3d0 msgr2=0x7fadf8108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.957+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 shutdown_connections 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:54.957+0000 7fadffc00700 1 -- 172.21.15.27:0/815030268 wait complete. 2024-04-26T20:58:55.898 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-04-26T20:58:56.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:55 smithi027 systemd[1]: Stopping Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T20:58:56.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a[27709]: 2024-04-26T20:58:56.385+0000 7f8617600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T20:58:56.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a[27709]: 2024-04-26T20:58:56.385+0000 7f8617600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T20:58:56.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 podman[27955]: 2024-04-26 20:58:56.427755039 +0000 UTC m=+0.214479836 container stop 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T20:58:56.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 podman[27955]: 2024-04-26 20:58:56.455911415 +0000 UTC m=+0.242636190 container died 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:58:57.121 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 podman[27955]: 2024-04-26 20:58:56.861629072 +0000 UTC m=+0.648353858 container cleanup 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T20:58:57.121 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:56 smithi027 bash[27955]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a 2024-04-26T20:58:57.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 podman[27969]: 2024-04-26 20:58:57.120752446 +0000 UTC m=+0.677161521 container remove 65634db5febc0b2fdf7932e1f710a9bd1da90950d0f2a7627a67938ad7b78368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T20:58:57.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T20:58:57.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 systemd[1]: Stopped Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T20:58:57.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.792s CPU time. 2024-04-26T20:58:57.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 systemd[1]: Starting Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T20:58:58.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 podman[28066]: 2024-04-26 20:58:57.792883406 +0000 UTC m=+0.034880372 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:58:58.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:57 smithi027 podman[28066]: 2024-04-26 20:58:57.915858726 +0000 UTC m=+0.157855712 container create 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:58:58.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 podman[28066]: 2024-04-26 20:58:58.167173432 +0000 UTC m=+0.409170424 container init 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T20:58:58.268 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 podman[28066]: 2024-04-26 20:58:58.178742735 +0000 UTC m=+0.420739689 container start 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: pidfile_write: ignore empty --pid-file 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: load: jerasure load: lrc 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: RocksDB version: 7.9.2 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Git sha 0 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: DB SUMMARY 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: DB Session ID: HFLQDP8KXA87BOGK7C39 2024-04-26T20:58:58.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: CURRENT file: CURRENT 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104878 ; 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.error_if_exists: 0 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.create_if_missing: 0 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.paranoid_checks: 1 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.env: 0x56247ec24cc0 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.info_log: 0x5624b7d57dc0 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.statistics: (nil) 2024-04-26T20:58:58.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.use_fsync: 0 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_log_file_size: 0 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T20:58:58.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_fallocate: 1 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.use_direct_reads: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.db_log_dir: 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.wal_dir: 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T20:58:58.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.write_buffer_manager: 0x5624b7d290e0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.unordered_write: 0 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T20:58:58.424 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.row_cache: None 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.wal_filter: None 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.two_write_queues: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.wal_compression: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.atomic_flush: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.log_readahead_size: 0 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T20:58:58.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_background_jobs: 2 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_background_compactions: -1 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_subcompactions: 1 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T20:58:58.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_open_files: -1 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_background_flushes: -1 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Compression algorithms supported: 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kZSTD supported: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kXpressCompression supported: 0 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kZlibCompression supported: 1 2024-04-26T20:58:58.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kSnappyCompression supported: 1 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kLZ4Compression supported: 1 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: kBZip2Compression supported: 0 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.merge_operator: 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_filter: None 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5624b7d57f80) 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-04-26T20:58:58.428 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x5624b7d5b1f0 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-04-26T20:58:58.429 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-04-26T20:58:58.430 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression: NoCompression 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.num_levels: 7 2024-04-26T20:58:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T20:58:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T20:58:58.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T20:58:58.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T20:58:58.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.inplace_update_support: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.bloom_locality: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.max_successive_merges: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.ttl: 2592000 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T20:58:58.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enable_blob_files: false 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.min_blob_size: 0 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2d004b18-ff58-46df-9a38-668e71a55d57 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165138238500, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T20:58:58.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165138240829, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100036, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97918, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14129, "raw_average_key_size": 49, "raw_value_size": 90254, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714165138, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2d004b18-ff58-46df-9a38-668e71a55d57", "db_session_id": "HFLQDP8KXA87BOGK7C39", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165138241003, "job": 1, "event": "recovery_finished"} 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5624b7e78000 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: DB pointer 0x5624b7e64000 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T20:58:58.438 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T20:58:58.439 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 13.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 13.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x5624b7d5b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:58.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???) e1 preinit fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).mds e1 new map 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).mds e1 print_map 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 bash[28066]: 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 systemd[1]: Started Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T20:58:58.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: monmap epoch 1 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: last_changed 2024-04-26T20:58:46.841103+0000 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: min_mon_release 19 (squid) 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: election_strategy: 1 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: fsmap 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T20:58:58.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:58:58 smithi027 ceph-mon[28085]: mgrmap e1: no daemons active 2024-04-26T20:58:59.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.975+0000 7f86f1e00700 1 Processor -- start 2024-04-26T20:58:59.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.975+0000 7f86f1e00700 1 -- start start 2024-04-26T20:58:59.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.976+0000 7f86f1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:59.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.976+0000 7f86f1e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec07cdf0 con 0x7f86ec1076f0 2024-04-26T20:58:59.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.977+0000 7f86eaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.977+0000 7f86eaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39530/0 (socket says 172.21.15.27:39530) 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.977+0000 7f86eaa00700 1 -- 172.21.15.27:0/3578114046 learned_addr learned my addr 172.21.15.27:0/3578114046 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.977+0000 7f86eaa00700 1 -- 172.21.15.27:0/3578114046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec108050 con 0x7f86ec1076f0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.978+0000 7f86eaa00700 1 --2- 172.21.15.27:0/3578114046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906cf959831cca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.978+0000 7f86e9600700 1 -- 172.21.15.27:0/3578114046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc004160 con 0x7f86ec1076f0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.978+0000 7f86e9600700 1 -- 172.21.15.27:0/3578114046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f86dc004300 con 0x7f86ec1076f0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.978+0000 7f86e9600700 1 -- 172.21.15.27:0/3578114046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc0119e0 con 0x7f86ec1076f0 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 -- 172.21.15.27:0/3578114046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 msgr2=0x7f86ec107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:59.754 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 --2- 172.21.15.27:0/3578114046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).stop 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 -- 172.21.15.27:0/3578114046 shutdown_connections 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 --2- 172.21.15.27:0/3578114046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec1076f0 0x7f86ec107b10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 -- 172.21.15.27:0/3578114046 >> 172.21.15.27:0/3578114046 conn(0x7f86ec1032b0 msgr2=0x7f86ec1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 -- 172.21.15.27:0/3578114046 shutdown_connections 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.979+0000 7f86f1e00700 1 -- 172.21.15.27:0/3578114046 wait complete. 2024-04-26T20:58:59.757 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.980+0000 7f86f1e00700 1 Processor -- start 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86f1e00700 1 -- start start 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86f1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86f1e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec109020 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86eaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86eaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39540/0 (socket says 172.21.15.27:39540) 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.981+0000 7f86eaa00700 1 -- 172.21.15.27:0/2768464380 learned_addr learned my addr 172.21.15.27:0/2768464380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86eaa00700 1 -- 172.21.15.27:0/2768464380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec19f9d0 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86eaa00700 1 --2- 172.21.15.27:0/2768464380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0116a0 tx=0x7f86dc012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc011a60 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f86dc003b10 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc01ace0 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec19fc60 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.982+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec1a2740 con 0x7f86ec19f070 2024-04-26T20:58:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.983+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f86dc02d7d0 con 0x7f86ec19f070 2024-04-26T20:58:59.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.983+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86dc024430 con 0x7f86ec19f070 2024-04-26T20:58:59.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.984+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec064a80 con 0x7f86ec19f070 2024-04-26T20:58:59.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:58.987+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f86dc043ec0 con 0x7f86ec19f070 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.064+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f86ec064030 con 0x7f86ec19f070 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.069+0000 7f86f0a00700 1 -- 172.21.15.27:0/2768464380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f86dc032030 con 0x7f86ec19f070 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 msgr2=0x7f86ec19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 --2- 172.21.15.27:0/2768464380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0116a0 tx=0x7f86dc012f80 comp rx=0 tx=0).stop 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 shutdown_connections 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 --2- 172.21.15.27:0/2768464380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec19f070 0x7f86ec19f490 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 >> 172.21.15.27:0/2768464380 conn(0x7f86ec1032b0 msgr2=0x7f86ec195b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 shutdown_connections 2024-04-26T20:58:59.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:58:59.070+0000 7f86f1e00700 1 -- 172.21.15.27:0/2768464380 wait complete. 2024-04-26T20:58:59.761 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T20:58:59.783 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T20:58:59.783 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-04-26T20:58:59.785 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T20:58:59.786 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T20:59:00.244 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x 2024-04-26T20:59:00.244 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T20:59:00.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2768464380' entity='client.admin' 2024-04-26T20:59:00.520 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968.target.wants/ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a125c556-040f-11ef-bc93-c7b262605968@.service. 2024-04-26T20:59:00.952 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:00 smithi027 systemd[1]: Starting Ceph mgr.x for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T20:59:01.235 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:00 smithi027 podman[28293]: 2024-04-26 20:59:00.857448621 +0000 UTC m=+0.041164767 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:59:01.236 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:00 smithi027 podman[28293]: 2024-04-26 20:59:00.96874926 +0000 UTC m=+0.152465453 container create eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T20:59:01.236 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 podman[28293]: 2024-04-26 20:59:01.189398058 +0000 UTC m=+0.373114255 container init eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T20:59:01.236 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 podman[28293]: 2024-04-26 20:59:01.199807473 +0000 UTC m=+0.383523666 container start eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T20:59:01.320 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T20:59:01.320 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T20:59:01.338 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T20:59:01.338 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T20:59:01.339 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T20:59:01.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T20:59:01.361 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T20:59:01.361 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-04-26T20:59:01.380 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-04-26T20:59:01.543 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 bash[28293]: eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d 2024-04-26T20:59:01.543 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 systemd[1]: Started Ceph mgr.x for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T20:59:01.543 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:01.453+0000 7faf9b643200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:59:01.843 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:01.719+0000 7faf9b643200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:02.094 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:01.858+0000 7faf9b643200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:59:02.421 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:02 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3148236630' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T20:59:02.421 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:02.301+0000 7faf9b643200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:59:02.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:02.419+0000 7faf9b643200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:59:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T20:59:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T20:59:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T20:58:51.281634+0000", 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.001+0000 7f9351200700 1 Processor -- start 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9351200700 1 -- start start 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9351200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9351200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f934c07d890 con 0x7f934c07ee60 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9349e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9349e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39564/0 (socket says 172.21.15.27:39564) 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.002+0000 7f9349e00700 1 -- 172.21.15.27:0/869135993 learned_addr learned my addr 172.21.15.27:0/869135993 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.003+0000 7f9349e00700 1 -- 172.21.15.27:0/869135993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c07da10 con 0x7f934c07ee60 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.003+0000 7f9349e00700 1 --2- 172.21.15.27:0/869135993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e42548a10a4737 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.004+0000 7f9348a00700 1 -- 172.21.15.27:0/869135993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c004160 con 0x7f934c07ee60 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.004+0000 7f9348a00700 1 -- 172.21.15.27:0/869135993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f933c004300 con 0x7f934c07ee60 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.004+0000 7f9348a00700 1 -- 172.21.15.27:0/869135993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c0119e0 con 0x7f934c07ee60 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.004+0000 7f9351200700 1 -- 172.21.15.27:0/869135993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 msgr2=0x7f934c07d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.004+0000 7f9351200700 1 --2- 172.21.15.27:0/869135993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.005+0000 7f9351200700 1 -- 172.21.15.27:0/869135993 shutdown_connections 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.005+0000 7f9351200700 1 --2- 172.21.15.27:0/869135993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c07d2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.005+0000 7f9351200700 1 -- 172.21.15.27:0/869135993 >> 172.21.15.27:0/869135993 conn(0x7f934c104350 msgr2=0x7f934c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.005+0000 7f9351200700 1 -- 172.21.15.27:0/869135993 shutdown_connections 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.005+0000 7f9351200700 1 -- 172.21.15.27:0/869135993 wait complete. 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.006+0000 7f9351200700 1 Processor -- start 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.006+0000 7f9351200700 1 -- start start 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.006+0000 7f9351200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.006+0000 7f9351200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f934c10f300 con 0x7f934c07ee60 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.006+0000 7f9349e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9349e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39572/0 (socket says 172.21.15.27:39572) 2024-04-26T20:59:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9349e00700 1 -- 172.21.15.27:0/3148236630 learned_addr learned my addr 172.21.15.27:0/3148236630 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9349e00700 1 -- 172.21.15.27:0/3148236630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c19f390 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9349e00700 1 --2- 172.21.15.27:0/3148236630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f933c0116a0 tx=0x7f933c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c003b70 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f933c01a8a0 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.007+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c024890 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.008+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c19f620 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.008+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c1a2310 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.008+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f933c022020 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.008+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f933c01a430 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.009+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c064a80 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.011+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f933c004300 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.091+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f934c1a2850 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.092+0000 7f9342a00700 1 -- 172.21.15.27:0/3148236630 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f933c01aa50 con 0x7f934c07ee60 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.093+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 msgr2=0x7f934c19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 --2- 172.21.15.27:0/3148236630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f933c0116a0 tx=0x7f933c0046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 shutdown_connections 2024-04-26T20:59:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 --2- 172.21.15.27:0/3148236630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f934c07ee60 0x7f934c19ee50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 >> 172.21.15.27:0/3148236630 conn(0x7f934c104350 msgr2=0x7f934c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 shutdown_connections 2024-04-26T20:59:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:02.094+0000 7f9351200700 1 -- 172.21.15.27:0/3148236630 wait complete. 2024-04-26T20:59:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-04-26T20:59:04.484 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:04.082+0000 7faf9b643200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:59:04.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:04.318+0000 7faf9b643200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:59:04.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:04.541+0000 7faf9b643200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:59:05.232 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:04.939+0000 7faf9b643200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:59:05.232 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:05.072+0000 7faf9b643200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:59:05.482 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:05.341+0000 7faf9b643200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:59:05.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:05 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2838736567' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T20:59:05.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:05.626+0000 7faf9b643200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:06.429 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:06.429 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T20:59:06.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T20:59:06.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T20:59:06.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T20:59:06.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T20:59:06.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T20:58:51.281634+0000", 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T20:59:06.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.389+0000 7fcc58000700 1 Processor -- start 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.390+0000 7fcc58000700 1 -- start start 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.390+0000 7fcc58000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.390+0000 7fcc58000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50109780 con 0x7fcc50108d90 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.391+0000 7fcc55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.391+0000 7fcc55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39586/0 (socket says 172.21.15.27:39586) 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.391+0000 7fcc55600700 1 -- 172.21.15.27:0/1293122617 learned_addr learned my addr 172.21.15.27:0/1293122617 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.391+0000 7fcc55600700 1 -- 172.21.15.27:0/1293122617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc50109f90 con 0x7fcc50108d90 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.392+0000 7fcc55600700 1 --2- 172.21.15.27:0/1293122617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eed23baca0f5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.392+0000 7fcc4fe00700 1 -- 172.21.15.27:0/1293122617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44004160 con 0x7fcc50108d90 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.392+0000 7fcc4fe00700 1 -- 172.21.15.27:0/1293122617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcc44004300 con 0x7fcc50108d90 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.392+0000 7fcc4fe00700 1 -- 172.21.15.27:0/1293122617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc440119e0 con 0x7fcc50108d90 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 -- 172.21.15.27:0/1293122617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 msgr2=0x7fcc501091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:06.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 --2- 172.21.15.27:0/1293122617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 -- 172.21.15.27:0/1293122617 shutdown_connections 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 --2- 172.21.15.27:0/1293122617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc501091b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 -- 172.21.15.27:0/1293122617 >> 172.21.15.27:0/1293122617 conn(0x7fcc50104350 msgr2=0x7fcc50106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 -- 172.21.15.27:0/1293122617 shutdown_connections 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.393+0000 7fcc58000700 1 -- 172.21.15.27:0/1293122617 wait complete. 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.394+0000 7fcc58000700 1 Processor -- start 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.394+0000 7fcc58000700 1 -- start start 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc58000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc58000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5010d1c0 con 0x7fcc50108d90 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc55600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39596/0 (socket says 172.21.15.27:39596) 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc55600700 1 -- 172.21.15.27:0/2838736567 learned_addr learned my addr 172.21.15.27:0/2838736567 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.395+0000 7fcc55600700 1 -- 172.21.15.27:0/2838736567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5019f3e0 con 0x7fcc50108d90 2024-04-26T20:59:06.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc55600700 1 --2- 172.21.15.27:0/2838736567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006b20 tx=0x7fcc44011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44003a60 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcc44003c00 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5019f670 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44024780 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501a2360 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc44022020 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.396+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc440193c0 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.397+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50064a80 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.400+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcc44042ec0 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.480+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc501a28b0 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.482+0000 7fcc4e000700 1 -- 172.21.15.27:0/2838736567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcc44031030 con 0x7fcc50108d90 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.483+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 msgr2=0x7fcc5019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.483+0000 7fcc58000700 1 --2- 172.21.15.27:0/2838736567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006b20 tx=0x7fcc44011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:06.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.483+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 shutdown_connections 2024-04-26T20:59:06.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.483+0000 7fcc58000700 1 --2- 172.21.15.27:0/2838736567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50108d90 0x7fcc5019eea0 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006b20 tx=0x7fcc44011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:06.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.483+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 >> 172.21.15.27:0/2838736567 conn(0x7fcc50104350 msgr2=0x7fcc50104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:06.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.484+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 shutdown_connections 2024-04-26T20:59:06.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:05.484+0000 7fcc58000700 1 -- 172.21.15.27:0/2838736567 wait complete. 2024-04-26T20:59:06.445 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-04-26T20:59:06.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:06.466+0000 7faf9b643200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:59:07.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:07.038+0000 7faf9b643200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:59:07.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:07.172+0000 7faf9b643200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:59:07.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:07.453+0000 7faf9b643200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:59:08.277 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:08.147+0000 7faf9b643200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:59:08.605 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:08.275+0000 7faf9b643200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:59:08.945 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:08.620+0000 7faf9b643200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:59:08.945 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:08.810+0000 7faf9b643200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:59:08.945 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:08.943+0000 7faf9b643200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:59:09.198 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:09.104+0000 7faf9b643200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:59:09.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3847372990' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T20:59:09.850 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:09.510+0000 7faf9b643200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:59:09.867 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:09.868 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T20:59:09.869 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T20:59:09.870 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T20:59:09.871 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T20:58:51.281634+0000", 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:09.872 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.105+0000 7fd262400700 1 Processor -- start 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd262400700 1 -- start start 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd262400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd262400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c109780 con 0x7fd25c108d90 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd25b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd25b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34492/0 (socket says 172.21.15.27:34492) 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.106+0000 7fd25b400700 1 -- 172.21.15.27:0/173071607 learned_addr learned my addr 172.21.15.27:0/173071607 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:09.873 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.107+0000 7fd25b400700 1 -- 172.21.15.27:0/173071607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c109f90 con 0x7fd25c108d90 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.107+0000 7fd25b400700 1 --2- 172.21.15.27:0/173071607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e8069c7a75b08a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.107+0000 7fd25a000700 1 -- 172.21.15.27:0/173071607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c004160 con 0x7fd25c108d90 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd25a000700 1 -- 172.21.15.27:0/173071607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd24c004300 con 0x7fd25c108d90 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd25a000700 1 -- 172.21.15.27:0/173071607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c0119e0 con 0x7fd25c108d90 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd262400700 1 -- 172.21.15.27:0/173071607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 msgr2=0x7fd25c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd262400700 1 --2- 172.21.15.27:0/173071607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd262400700 1 -- 172.21.15.27:0/173071607 shutdown_connections 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd262400700 1 --2- 172.21.15.27:0/173071607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c1091b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.108+0000 7fd262400700 1 -- 172.21.15.27:0/173071607 >> 172.21.15.27:0/173071607 conn(0x7fd25c104350 msgr2=0x7fd25c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.109+0000 7fd262400700 1 -- 172.21.15.27:0/173071607 shutdown_connections 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.109+0000 7fd262400700 1 -- 172.21.15.27:0/173071607 wait complete. 2024-04-26T20:59:09.874 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd262400700 1 Processor -- start 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd262400700 1 -- start start 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd262400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd262400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c10d1c0 con 0x7fd25c108d90 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd25b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd25b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34500/0 (socket says 172.21.15.27:34500) 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.110+0000 7fd25b400700 1 -- 172.21.15.27:0/3847372990 learned_addr learned my addr 172.21.15.27:0/3847372990 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd25b400700 1 -- 172.21.15.27:0/3847372990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c19d1e0 con 0x7fd25c108d90 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd25b400700 1 --2- 172.21.15.27:0/3847372990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0119b0 tx=0x7fd24c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c003b70 con 0x7fd25c108d90 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd24c01a8a0 con 0x7fd25c108d90 2024-04-26T20:59:09.875 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c024830 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.111+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c19d470 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.112+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c1a0160 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.112+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd24c022020 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.112+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd24c019610 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.113+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c064a80 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.115+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd24c01aa50 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.197+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd25c1a0330 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.197+0000 7fd253e00700 1 -- 172.21.15.27:0/3847372990 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd24c01a430 con 0x7fd25c108d90 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.199+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 msgr2=0x7fd25c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.199+0000 7fd262400700 1 --2- 172.21.15.27:0/3847372990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0119b0 tx=0x7fd24c011a90 comp rx=0 tx=0).stop 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.200+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 shutdown_connections 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.200+0000 7fd262400700 1 --2- 172.21.15.27:0/3847372990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd25c108d90 0x7fd25c19cca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:09.876 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.200+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 >> 172.21.15.27:0/3847372990 conn(0x7fd25c104350 msgr2=0x7fd25c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:09.877 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.200+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 shutdown_connections 2024-04-26T20:59:09.877 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:09.200+0000 7fd262400700 1 -- 172.21.15.27:0/3847372990 wait complete. 2024-04-26T20:59:09.877 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2024-04-26T20:59:10.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:10.315+0000 7faf9b643200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:10.615+0000 7faf9b643200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: Activating manager daemon x 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: mgrmap e2: x(active, starting, since 0.011382s) 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: Manager daemon x is now available 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' 2024-04-26T20:59:10.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T20:59:10.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' 2024-04-26T20:59:10.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:10 smithi027 ceph-mon[28085]: from='mgr.14100 172.21.15.27:0/3635680186' entity='mgr.x' 2024-04-26T20:59:12.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:12 smithi027 ceph-mon[28085]: mgrmap e3: x(active, since 1.02252s) 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T20:59:13.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T20:59:13.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T20:59:13.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:13.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T20:59:13.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T20:59:13.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T20:59:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T20:59:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T20:59:13.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T20:58:51.281634+0000", 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.520+0000 7f734be00700 1 Processor -- start 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.520+0000 7f734be00700 1 -- start start 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.521+0000 7f734be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:13.798 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.521+0000 7f734be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7344107a80 con 0x7f7344107090 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.521+0000 7f7349400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.521+0000 7f7349400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34580/0 (socket says 172.21.15.27:34580) 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.521+0000 7f7349400700 1 -- 172.21.15.27:0/1838603198 learned_addr learned my addr 172.21.15.27:0/1838603198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.522+0000 7f7349400700 1 -- 172.21.15.27:0/1838603198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344107c00 con 0x7f7344107090 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.522+0000 7f7349400700 1 --2- 172.21.15.27:0/1838603198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f73380098a0 tx=0x7f7338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9abc968d3bc631e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.522+0000 7f7343e00700 1 -- 172.21.15.27:0/1838603198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7338004160 con 0x7f7344107090 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.523+0000 7f7343e00700 1 -- 172.21.15.27:0/1838603198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7338004300 con 0x7f7344107090 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.523+0000 7f7343e00700 1 -- 172.21.15.27:0/1838603198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73380119e0 con 0x7f7344107090 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.523+0000 7f734be00700 1 -- 172.21.15.27:0/1838603198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 msgr2=0x7f73441074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.523+0000 7f734be00700 1 --2- 172.21.15.27:0/1838603198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f73380098a0 tx=0x7f7338009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.524+0000 7f734be00700 1 -- 172.21.15.27:0/1838603198 shutdown_connections 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.524+0000 7f734be00700 1 --2- 172.21.15.27:0/1838603198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f73441074b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.524+0000 7f734be00700 1 -- 172.21.15.27:0/1838603198 >> 172.21.15.27:0/1838603198 conn(0x7f7344102a30 msgr2=0x7f7344104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:13.799 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.524+0000 7f734be00700 1 -- 172.21.15.27:0/1838603198 shutdown_connections 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.524+0000 7f734be00700 1 -- 172.21.15.27:0/1838603198 wait complete. 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.525+0000 7f734be00700 1 Processor -- start 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.525+0000 7f734be00700 1 -- start start 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.525+0000 7f734be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.525+0000 7f734be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f734410af10 con 0x7f7344107090 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.526+0000 7f7349400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.526+0000 7f7349400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34584/0 (socket says 172.21.15.27:34584) 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.526+0000 7f7349400700 1 -- 172.21.15.27:0/3177263563 learned_addr learned my addr 172.21.15.27:0/3177263563 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.526+0000 7f7349400700 1 -- 172.21.15.27:0/3177263563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344109ed0 con 0x7f7344107090 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.526+0000 7f7349400700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7338006b20 tx=0x7f7338011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.527+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7338003a60 con 0x7f7344107090 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.527+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7338003c00 con 0x7f7344107090 2024-04-26T20:59:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.527+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73441080c0 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.527+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7338024780 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.527+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7344108600 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.528+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f7338022020 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.528+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344064a80 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.529+0000 7f7342000700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 0x7f732c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.529+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f733804d4b0 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.529+0000 7f7348a00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 0x7f732c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.530+0000 7f7348a00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 0x7f732c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7330006fd0 tx=0x7f7330006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.536+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f733801a950 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.642+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f733801a430 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.860+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7344108880 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.861+0000 7f7342000700 1 -- 172.21.15.27:0/3177263563 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7338044650 con 0x7f7344107090 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 msgr2=0x7f732c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 0x7f732c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7330006fd0 tx=0x7f7330006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 msgr2=0x7f7344109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7338006b20 tx=0x7f7338011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 shutdown_connections 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f732c039490 0x7f732c03b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 --2- 172.21.15.27:0/3177263563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7344107090 0x7f7344109990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 >> 172.21.15.27:0/3177263563 conn(0x7f7344102a30 msgr2=0x7f7344103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 shutdown_connections 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:12.866+0000 7f734be00700 1 -- 172.21.15.27:0/3177263563 wait complete. 2024-04-26T20:59:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-04-26T20:59:13.942 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:13 smithi027 ceph-mon[28085]: mgrmap e4: x(active, since 2s) 2024-04-26T20:59:13.942 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:13 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3177263563' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T20:59:14.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:14 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4123298316' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T20:59:15.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T20:59:15.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T20:59:15.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T20:59:15.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:15.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T20:59:15.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T20:59:15.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.420+0000 7fd714400700 1 Processor -- start 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.420+0000 7fd714400700 1 -- start start 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.421+0000 7fd714400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.421+0000 7fd714400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c07cdb0 con 0x7fd70c109900 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.421+0000 7fd711a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.421+0000 7fd711a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34586/0 (socket says 172.21.15.27:34586) 2024-04-26T20:59:15.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.421+0000 7fd711a00700 1 -- 172.21.15.27:0/3861967029 learned_addr learned my addr 172.21.15.27:0/3861967029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.422+0000 7fd711a00700 1 -- 172.21.15.27:0/3861967029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c10c230 con 0x7fd70c109900 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.422+0000 7fd711a00700 1 --2- 172.21.15.27:0/3861967029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098a0 tx=0x7fd700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e4b1c0831245f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.422+0000 7fd70be00700 1 -- 172.21.15.27:0/3861967029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700004160 con 0x7fd70c109900 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.422+0000 7fd70be00700 1 -- 172.21.15.27:0/3861967029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd700004300 con 0x7fd70c109900 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.422+0000 7fd70be00700 1 -- 172.21.15.27:0/3861967029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7000119e0 con 0x7fd70c109900 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 -- 172.21.15.27:0/3861967029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 msgr2=0x7fd70c10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 --2- 172.21.15.27:0/3861967029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098a0 tx=0x7fd700009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 -- 172.21.15.27:0/3861967029 shutdown_connections 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 --2- 172.21.15.27:0/3861967029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c10bcf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:15.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 -- 172.21.15.27:0/3861967029 >> 172.21.15.27:0/3861967029 conn(0x7fd70c103270 msgr2=0x7fd70c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.423+0000 7fd714400700 1 -- 172.21.15.27:0/3861967029 shutdown_connections 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.424+0000 7fd714400700 1 -- 172.21.15.27:0/3861967029 wait complete. 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.425+0000 7fd714400700 1 Processor -- start 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.425+0000 7fd714400700 1 -- start start 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.425+0000 7fd714400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.425+0000 7fd714400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c10d0d0 con 0x7fd70c109900 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.426+0000 7fd711a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.426+0000 7fd711a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34596/0 (socket says 172.21.15.27:34596) 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.426+0000 7fd711a00700 1 -- 172.21.15.27:0/4123298316 learned_addr learned my addr 172.21.15.27:0/4123298316 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.426+0000 7fd711a00700 1 -- 172.21.15.27:0/4123298316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c19f420 con 0x7fd70c109900 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd711a00700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7000116a0 tx=0x7fd7000046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700003b70 con 0x7fd70c109900 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd70001a8a0 con 0x7fd70c109900 2024-04-26T20:59:15.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c19f6b0 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700024890 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.427+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c1a2350 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.428+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd700022020 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.428+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c064a80 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.429+0000 7fd70a000700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 0x7fd6f403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.429+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd70004d7f0 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.429+0000 7fd711000700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 0x7fd6f403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.430+0000 7fd711000700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 0x7fd6f403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8006fd0 tx=0x7fd6f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.435+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd70000b3e0 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.662+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd70c1a2520 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.665+0000 7fd70a000700 1 -- 172.21.15.27:0/4123298316 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fd70c1a2520 con 0x7fd70c109900 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 msgr2=0x7fd6f403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 0x7fd6f403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8006fd0 tx=0x7fd6f8006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 msgr2=0x7fd70c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7000116a0 tx=0x7fd7000046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:15.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 shutdown_connections 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fd6f40395b0 0x7fd6f403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 --2- 172.21.15.27:0/4123298316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd70c109900 0x7fd70c19eee0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 >> 172.21.15.27:0/4123298316 conn(0x7fd70c103270 msgr2=0x7fd70c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.668+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 shutdown_connections 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:14.669+0000 7fd714400700 1 -- 172.21.15.27:0/4123298316 wait complete. 2024-04-26T20:59:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-04-26T20:59:16.624 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:16 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2569905713' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T20:59:16.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setuser ceph since I am not root 2024-04-26T20:59:16.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setgroup ceph since I am not root 2024-04-26T20:59:16.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:16.622+0000 7fbb163c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:59:17.025 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.978+0000 7f03a4600700 1 Processor -- start 2024-04-26T20:59:17.025 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.978+0000 7f03a4600700 1 -- start start 2024-04-26T20:59:17.025 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.979+0000 7f03a4600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:17.030 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.979+0000 7f03a4600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f039c07ce10 con 0x7f039c1098a0 2024-04-26T20:59:17.030 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.980+0000 7f03a1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.980+0000 7f03a1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34606/0 (socket says 172.21.15.27:34606) 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.980+0000 7f03a1c00700 1 -- 172.21.15.27:0/3026965658 learned_addr learned my addr 172.21.15.27:0/3026965658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.980+0000 7f03a1c00700 1 -- 172.21.15.27:0/3026965658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c10c1d0 con 0x7f039c1098a0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.981+0000 7f03a1c00700 1 --2- 172.21.15.27:0/3026965658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0390009e80 tx=0x7f039000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147d8f73acb3e88e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.981+0000 7f039be00700 1 -- 172.21.15.27:0/3026965658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390017070 con 0x7f039c1098a0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.981+0000 7f039be00700 1 -- 172.21.15.27:0/3026965658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f039000b8c0 con 0x7f039c1098a0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.981+0000 7f039be00700 1 -- 172.21.15.27:0/3026965658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390004420 con 0x7f039c1098a0 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 -- 172.21.15.27:0/3026965658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 msgr2=0x7f039c10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 --2- 172.21.15.27:0/3026965658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0390009e80 tx=0x7f039000b330 comp rx=0 tx=0).stop 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 -- 172.21.15.27:0/3026965658 shutdown_connections 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 --2- 172.21.15.27:0/3026965658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c10bc90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 -- 172.21.15.27:0/3026965658 >> 172.21.15.27:0/3026965658 conn(0x7f039c103270 msgr2=0x7f039c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:17.031 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 -- 172.21.15.27:0/3026965658 shutdown_connections 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.982+0000 7f03a4600700 1 -- 172.21.15.27:0/3026965658 wait complete. 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.983+0000 7f03a4600700 1 Processor -- start 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.983+0000 7f03a4600700 1 -- start start 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a4600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a4600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f039c10d0d0 con 0x7f039c1098a0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34612/0 (socket says 172.21.15.27:34612) 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a1c00700 1 -- 172.21.15.27:0/2569905713 learned_addr learned my addr 172.21.15.27:0/2569905713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.984+0000 7f03a1c00700 1 -- 172.21.15.27:0/2569905713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c19d1d0 con 0x7f039c1098a0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f03a1c00700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0390000f80 tx=0x7f0390003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390020020 con 0x7f039c1098a0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0390003e20 con 0x7f039c1098a0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390017040 con 0x7f039c1098a0 2024-04-26T20:59:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c19d460 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.985+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a0150 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.986+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0390023020 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.986+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c064a80 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.987+0000 7f039a000700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 0x7f038403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.987+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f039000f050 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.987+0000 7f03a1200700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 0x7f038403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.988+0000 7f03a1200700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 0x7f038403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0388006fd0 tx=0x7f0388006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:15.992+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f039002a020 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.283+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f039c1a0840 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.337+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f039002c300 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.340+0000 7f039a000700 1 -- 172.21.15.27:0/2569905713 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0390012430 con 0x7f039c1098a0 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 msgr2=0x7f038403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:17.033 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 0x7f038403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0388006fd0 tx=0x7f0388006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:17.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 msgr2=0x7f039c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0390000f80 tx=0x7f0390003ad0 comp rx=0 tx=0).stop 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 shutdown_connections 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f0384039600 0x7f038403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 --2- 172.21.15.27:0/2569905713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c1098a0 0x7f039c19cc90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.343+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 >> 172.21.15.27:0/2569905713 conn(0x7f039c103270 msgr2=0x7f039c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.344+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 shutdown_connections 2024-04-26T20:59:17.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:16.344+0000 7f03a4600700 1 -- 172.21.15.27:0/2569905713 wait complete. 2024-04-26T20:59:17.180 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:16.875+0000 7fbb163c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:17.180 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:17.012+0000 7fbb163c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:59:17.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:17 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2569905713' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T20:59:17.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:17 smithi027 ceph-mon[28085]: mgrmap e5: x(active, since 5s) 2024-04-26T20:59:17.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:17.432+0000 7fbb163c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:59:17.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:17.551+0000 7fbb163c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:59:18.649 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:18 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1981927155' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T20:59:18.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:18.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T20:59:18.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.614+0000 7fcd3b600700 1 Processor -- start 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.614+0000 7fcd3b600700 1 -- start start 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.615+0000 7fcd3b600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:18.667 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.615+0000 7fcd3b600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3407cdb0 con 0x7fcd34109900 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.615+0000 7fcd38c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.615+0000 7fcd38c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34644/0 (socket says 172.21.15.27:34644) 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.615+0000 7fcd38c00700 1 -- 172.21.15.27:0/1340557176 learned_addr learned my addr 172.21.15.27:0/1340557176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.616+0000 7fcd38c00700 1 -- 172.21.15.27:0/1340557176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3410c230 con 0x7fcd34109900 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.616+0000 7fcd38c00700 1 --2- 172.21.15.27:0/1340557176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd280098a0 tx=0x7fcd28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adec689a3ff13a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.616+0000 7fcd33400700 1 -- 172.21.15.27:0/1340557176 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd28004160 con 0x7fcd34109900 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.616+0000 7fcd33400700 1 -- 172.21.15.27:0/1340557176 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcd28004300 con 0x7fcd34109900 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.616+0000 7fcd33400700 1 -- 172.21.15.27:0/1340557176 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd280119e0 con 0x7fcd34109900 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 -- 172.21.15.27:0/1340557176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 msgr2=0x7fcd3410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1340557176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd280098a0 tx=0x7fcd28009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 -- 172.21.15.27:0/1340557176 shutdown_connections 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1340557176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3410bcf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 -- 172.21.15.27:0/1340557176 >> 172.21.15.27:0/1340557176 conn(0x7fcd34103270 msgr2=0x7fcd34105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.617+0000 7fcd3b600700 1 -- 172.21.15.27:0/1340557176 shutdown_connections 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.618+0000 7fcd3b600700 1 -- 172.21.15.27:0/1340557176 wait complete. 2024-04-26T20:59:18.668 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd3b600700 1 Processor -- start 2024-04-26T20:59:18.669 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd3b600700 1 -- start start 2024-04-26T20:59:18.669 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd3b600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:18.669 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd3b600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3410d0d0 con 0x7fcd34109900 2024-04-26T20:59:18.669 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd38c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd38c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34650/0 (socket says 172.21.15.27:34650) 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.619+0000 7fcd38c00700 1 -- 172.21.15.27:0/1981927155 learned_addr learned my addr 172.21.15.27:0/1981927155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd38c00700 1 -- 172.21.15.27:0/1981927155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3419d190 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd38c00700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd280116a0 tx=0x7fcd280046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd28003b70 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcd2801a8a0 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd28024890 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.620+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3419d420 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.621+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a0110 con 0x7fcd34109900 2024-04-26T20:59:18.672 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.622+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd34064a80 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.622+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcd28022020 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.622+0000 7fcd31600700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 0x7fcd1c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.622+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd2804d790 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.622+0000 7fcd33e00700 1 -- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 msgr2=0x7fcd1c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.623+0000 7fcd33e00700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 0x7fcd1c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.628+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcd2800f2e0 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.824+0000 7fcd33e00700 1 -- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 msgr2=0x7fcd1c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.824+0000 7fcd33e00700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 0x7fcd1c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.913+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcd341a02e0 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.914+0000 7fcd31600700 1 -- 172.21.15.27:0/1981927155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcd28046e20 con 0x7fcd34109900 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.916+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 msgr2=0x7fcd1c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.916+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 0x7fcd1c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.916+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 msgr2=0x7fcd3419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.916+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd280116a0 tx=0x7fcd280046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 shutdown_connections 2024-04-26T20:59:18.673 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7fcd1c039600 0x7fcd1c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 --2- 172.21.15.27:0/1981927155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd34109900 0x7fcd3419cc50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 >> 172.21.15.27:0/1981927155 conn(0x7fcd34103270 msgr2=0x7fcd34103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 shutdown_connections 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:17.917+0000 7fcd3b600700 1 -- 172.21.15.27:0/1981927155 wait complete. 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-04-26T20:59:18.674 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-04-26T20:59:19.484 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:19.217+0000 7fbb163c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:59:19.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:19.453+0000 7fbb163c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:59:19.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:19.679+0000 7fbb163c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:59:20.469 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:20.074+0000 7fbb163c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:59:20.469 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:20.202+0000 7fbb163c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:59:20.733 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:20.467+0000 7fbb163c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:59:21.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:20.732+0000 7fbb163c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:21.985 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:21.561+0000 7fbb163c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:59:22.484 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:22.136+0000 7fbb163c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:59:22.484 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:22.272+0000 7fbb163c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:59:22.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:22.549+0000 7fbb163c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:59:23.726 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:23.253+0000 7fbb163c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:59:23.726 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:23.385+0000 7fbb163c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:59:23.727 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:23.724+0000 7fbb163c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:59:23.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:23.912+0000 7fbb163c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:59:24.484 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:24.047+0000 7fbb163c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:59:24.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:24.205+0000 7fbb163c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:59:24.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:24.600+0000 7fbb163c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:59:25.695 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:25.394+0000 7fbb163c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:25.693+0000 7fbb163c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: Active manager daemon x restarted 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: Activating manager daemon x 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: mgrmap e6: x(active, starting, since 0.00501178s) 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T20:59:25.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T20:59:25.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T20:59:25.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: Manager daemon x is now available 2024-04-26T20:59:25.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:25 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: Found migration_current of "None". Setting to last migration. 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:26 smithi027 ceph-mon[28085]: mgrmap e7: x(active, since 1.01103s) 2024-04-26T20:59:27.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:27.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T20:59:27.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T20:59:27.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:27.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.210+0000 7f605c400700 1 Processor -- start 2024-04-26T20:59:27.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.210+0000 7f605c400700 1 -- start start 2024-04-26T20:59:27.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.211+0000 7f605c400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:27.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.211+0000 7f605c400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f605407ce90 con 0x7f6054109820 2024-04-26T20:59:27.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.211+0000 7f6059a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:27.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.211+0000 7f6059a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38796/0 (socket says 172.21.15.27:38796) 2024-04-26T20:59:27.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.211+0000 7f6059a00700 1 -- 172.21.15.27:0/3260139666 learned_addr learned my addr 172.21.15.27:0/3260139666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.212+0000 7f6059a00700 1 -- 172.21.15.27:0/3260139666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605410c150 con 0x7f6054109820 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.212+0000 7f6059a00700 1 --2- 172.21.15.27:0/3260139666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60480098a0 tx=0x7f6048009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff78a08c931da3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.212+0000 7f6053e00700 1 -- 172.21.15.27:0/3260139666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6048004160 con 0x7f6054109820 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.212+0000 7f6053e00700 1 -- 172.21.15.27:0/3260139666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6048004300 con 0x7f6054109820 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f6053e00700 1 -- 172.21.15.27:0/3260139666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60480119e0 con 0x7f6054109820 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f605c400700 1 -- 172.21.15.27:0/3260139666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 msgr2=0x7f605410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f605c400700 1 --2- 172.21.15.27:0/3260139666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60480098a0 tx=0x7f6048009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f605c400700 1 -- 172.21.15.27:0/3260139666 shutdown_connections 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f605c400700 1 --2- 172.21.15.27:0/3260139666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605410bc10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.213+0000 7f605c400700 1 -- 172.21.15.27:0/3260139666 >> 172.21.15.27:0/3260139666 conn(0x7f6054103270 msgr2=0x7f6054105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.214+0000 7f605c400700 1 -- 172.21.15.27:0/3260139666 shutdown_connections 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.214+0000 7f605c400700 1 -- 172.21.15.27:0/3260139666 wait complete. 2024-04-26T20:59:27.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.215+0000 7f605c400700 1 Processor -- start 2024-04-26T20:59:27.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.215+0000 7f605c400700 1 -- start start 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.215+0000 7f605c400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.215+0000 7f605c400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f605410d0c0 con 0x7f6054109820 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.216+0000 7f6059a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.216+0000 7f6059a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38800/0 (socket says 172.21.15.27:38800) 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.216+0000 7f6059a00700 1 -- 172.21.15.27:0/3043154992 learned_addr learned my addr 172.21.15.27:0/3043154992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.216+0000 7f6059a00700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605419f3a0 con 0x7f6054109820 2024-04-26T20:59:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.216+0000 7f6059a00700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60480116a0 tx=0x7f60480046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.217+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6048003b70 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.217+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f604801a8a0 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.217+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6048024890 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.217+0000 7f605c400700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605419f630 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.217+0000 7f605c400700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60541a2320 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.218+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6048022020 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.219+0000 7f6052000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.219+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f603c03c1d0 con 0x7f603c039600 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.219+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f604804d7b0 con 0x7f6054109820 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.219+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.219+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.420+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.420+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.821+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:19.821+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:20.622+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:20.622+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:22.224+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:22.224+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:25.428+0000 7f6059000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2094082344 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:25.428+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:25.701+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f604801a430 con 0x7f6054109820 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:25.702+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 msgr2=0x7f603c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:25.702+0000 7f6052000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.706+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f604804f5f0 con 0x7f6054109820 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.706+0000 7f6052000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 0x7f603c03f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.707+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6048046c20 con 0x7f603c03cef0 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.709+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 0x7f603c03f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:27.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.712+0000 7f6059000700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 0x7f603c03f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6040003a10 tx=0x7f60400092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.715+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6048046c20 con 0x7f603c03cef0 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.725+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6054064a80 con 0x7f603c03cef0 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.726+0000 7f6052000700 1 -- 172.21.15.27:0/3043154992 <== mgr.14120 v2:172.21.15.27:6800/128242469 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6054064a80 con 0x7f603c03cef0 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.726+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 msgr2=0x7f603c03f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.726+0000 7f6047400700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 0x7f603c03f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6040003a10 tx=0x7f60400092b0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.726+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 msgr2=0x7f605419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.726+0000 7f6047400700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60480116a0 tx=0x7f60480046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 shutdown_connections 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6054109820 0x7f605419ee60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f603c03cef0 0x7f603c03f2e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 --2- 172.21.15.27:0/3043154992 >> [v2:172.21.15.27:6800/2094082344,v1:172.21.15.27:6801/2094082344] conn(0x7f603c039600 0x7f603c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 >> 172.21.15.27:0/3043154992 conn(0x7f6054103270 msgr2=0x7f6054103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 shutdown_connections 2024-04-26T20:59:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:26.727+0000 7f6047400700 1 -- 172.21.15.27:0/3043154992 wait complete. 2024-04-26T20:59:27.472 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-04-26T20:59:27.472 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-04-26T20:59:28.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T20:59:28.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T20:59:28.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: [26/Apr/2024:20:59:27] ENGINE Bus STARTING 2024-04-26T20:59:28.024 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: [26/Apr/2024:20:59:27] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T20:59:28.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: [26/Apr/2024:20:59:27] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T20:59:28.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: [26/Apr/2024:20:59:27] ENGINE Bus STARTED 2024-04-26T20:59:28.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:27 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:28 smithi027 ceph-mon[28085]: mgrmap e8: x(active, since 2s) 2024-04-26T20:59:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:28 smithi027 ceph-mon[28085]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:28 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:28 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:29.086 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.069+0000 7f910f400700 1 Processor -- start 2024-04-26T20:59:29.090 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.070+0000 7f910f400700 1 -- start start 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.070+0000 7f910f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.070+0000 7f910f400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f910807d8f0 con 0x7f910807eec0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.070+0000 7f910ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.071+0000 7f910ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38860/0 (socket says 172.21.15.27:38860) 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.071+0000 7f910ca00700 1 -- 172.21.15.27:0/1646761183 learned_addr learned my addr 172.21.15.27:0/1646761183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.071+0000 7f910ca00700 1 -- 172.21.15.27:0/1646761183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910807da70 con 0x7f910807eec0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.071+0000 7f910ca00700 1 --2- 172.21.15.27:0/1646761183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0098a0 tx=0x7f90fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5a2c006095bbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.071+0000 7f9107400700 1 -- 172.21.15.27:0/1646761183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc004160 con 0x7f910807eec0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f9107400700 1 -- 172.21.15.27:0/1646761183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f90fc004300 con 0x7f910807eec0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f9107400700 1 -- 172.21.15.27:0/1646761183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc0119e0 con 0x7f910807eec0 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 -- 172.21.15.27:0/1646761183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 msgr2=0x7f910807d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:29.091 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 --2- 172.21.15.27:0/1646761183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0098a0 tx=0x7f90fc009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 -- 172.21.15.27:0/1646761183 shutdown_connections 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 --2- 172.21.15.27:0/1646761183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f910807d320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 -- 172.21.15.27:0/1646761183 >> 172.21.15.27:0/1646761183 conn(0x7f91081043b0 msgr2=0x7f91081067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 -- 172.21.15.27:0/1646761183 shutdown_connections 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.072+0000 7f910f400700 1 -- 172.21.15.27:0/1646761183 wait complete. 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.073+0000 7f910f400700 1 Processor -- start 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910f400700 1 -- start start 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910f400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f910810af00 con 0x7f910807eec0 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38862/0 (socket says 172.21.15.27:38862) 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910ca00700 1 -- 172.21.15.27:0/1461393907 learned_addr learned my addr 172.21.15.27:0/1461393907 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.074+0000 7f910ca00700 1 -- 172.21.15.27:0/1461393907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91081a37e0 con 0x7f910807eec0 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f910ca00700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90fc006b20 tx=0x7f90fc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:29.092 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc003a60 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f90fc003c00 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc024780 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081a3a70 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.075+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081a6760 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.076+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f90fc022020 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.076+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9108064a80 con 0x7f910807eec0 2024-04-26T20:59:29.093 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.077+0000 7f9105600700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 0x7f90f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.077+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f90fc04db30 con 0x7f910807eec0 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.077+0000 7f9107e00700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 0x7f90f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.078+0000 7f9107e00700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 0x7f90f003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90f8006fd0 tx=0x7f90f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.083+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f90fc00f2e0 con 0x7f910807eec0 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.322+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f91080008d0 con 0x7f90f0039600 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.334+0000 7f9105600700 1 -- 172.21.15.27:0/1461393907 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f91080008d0 con 0x7f90f0039600 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 msgr2=0x7f90f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 0x7f90f003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90f8006fd0 tx=0x7f90f8006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 msgr2=0x7f91081a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90fc006b20 tx=0x7f90fc011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 shutdown_connections 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f90f0039600 0x7f90f003bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 --2- 172.21.15.27:0/1461393907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f910807eec0 0x7f91081a32a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.337+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 >> 172.21.15.27:0/1461393907 conn(0x7f91081043b0 msgr2=0x7f91081067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.338+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 shutdown_connections 2024-04-26T20:59:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:28.338+0000 7f910f400700 1 -- 172.21.15.27:0/1461393907 wait complete. 2024-04-26T20:59:30.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:30 smithi027 ceph-mon[28085]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:30.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T20:59:30.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.743+0000 7f8323c00700 1 Processor -- start 2024-04-26T20:59:30.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.744+0000 7f8323c00700 1 -- start start 2024-04-26T20:59:30.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.744+0000 7f8323c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:30.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.744+0000 7f8323c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f831c07d8d0 con 0x7f831c07eea0 2024-04-26T20:59:30.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.745+0000 7f8321200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.745+0000 7f8321200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58794/0 (socket says 172.21.15.27:58794) 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.745+0000 7f8321200700 1 -- 172.21.15.27:0/191944004 learned_addr learned my addr 172.21.15.27:0/191944004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.745+0000 7f8321200700 1 -- 172.21.15.27:0/191944004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c07da50 con 0x7f831c07eea0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.745+0000 7f8321200700 1 --2- 172.21.15.27:0/191944004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83100098a0 tx=0x7f8310009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afb539f68722e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.746+0000 7f831b400700 1 -- 172.21.15.27:0/191944004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8310004160 con 0x7f831c07eea0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.746+0000 7f831b400700 1 -- 172.21.15.27:0/191944004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8310004300 con 0x7f831c07eea0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.746+0000 7f831b400700 1 -- 172.21.15.27:0/191944004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83100119e0 con 0x7f831c07eea0 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.746+0000 7f8323c00700 1 -- 172.21.15.27:0/191944004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 msgr2=0x7f831c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.746+0000 7f8323c00700 1 --2- 172.21.15.27:0/191944004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83100098a0 tx=0x7f8310009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.747+0000 7f8323c00700 1 -- 172.21.15.27:0/191944004 shutdown_connections 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.747+0000 7f8323c00700 1 --2- 172.21.15.27:0/191944004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c07d300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.747+0000 7f8323c00700 1 -- 172.21.15.27:0/191944004 >> 172.21.15.27:0/191944004 conn(0x7f831c104350 msgr2=0x7f831c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.747+0000 7f8323c00700 1 -- 172.21.15.27:0/191944004 shutdown_connections 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.747+0000 7f8323c00700 1 -- 172.21.15.27:0/191944004 wait complete. 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.748+0000 7f8323c00700 1 Processor -- start 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.748+0000 7f8323c00700 1 -- start start 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.748+0000 7f8323c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.748+0000 7f8323c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10f2c0 con 0x7f831c07eea0 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8321200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8321200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58798/0 (socket says 172.21.15.27:58798) 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8321200700 1 -- 172.21.15.27:0/3166709673 learned_addr learned my addr 172.21.15.27:0/3166709673 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8321200700 1 -- 172.21.15.27:0/3166709673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c19f3a0 con 0x7f831c07eea0 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8321200700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8310006b20 tx=0x7f8310011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:30.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8310003a60 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8310003c00 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.749+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c19f630 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.750+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8310024780 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.750+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a2320 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.751+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8310022020 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.751+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c064a80 con 0x7f831c07eea0 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.751+0000 7f8319600700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 0x7f830403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:30.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.751+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f831004da50 con 0x7f831c07eea0 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.752+0000 7f831be00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 0x7f830403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.752+0000 7f831be00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 0x7f830403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.757+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f831000f2e0 con 0x7f831c07eea0 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.986+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f831c105500 con 0x7f8304039600 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.987+0000 7f8319600700 1 -- 172.21.15.27:0/3166709673 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f831c105500 con 0x7f8304039600 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 msgr2=0x7f830403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 0x7f830403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 msgr2=0x7f831c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8310006b20 tx=0x7f8310011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 shutdown_connections 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f8304039600 0x7f830403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 --2- 172.21.15.27:0/3166709673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c07eea0 0x7f831c19ee60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.990+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 >> 172.21.15.27:0/3166709673 conn(0x7f831c104350 msgr2=0x7f831c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:30.618 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.991+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 shutdown_connections 2024-04-26T20:59:30.619 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:29.991+0000 7f8323c00700 1 -- 172.21.15.27:0/3166709673 wait complete. 2024-04-26T20:59:30.619 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-04-26T20:59:31.985 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: Generating public/private rsa key pair. 2024-04-26T20:59:31.985 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: Your identification has been saved in /tmp/tmpc5142w5p/key. 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: Your public key has been saved in /tmp/tmpc5142w5p/key.pub. 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: The key fingerprint is: 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: SHA256:jkcCFYg6h2zgHQ+vw5xFuNjWDNC0Nhr8r8DS9zPbxdU ceph-a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: The key's randomart image is: 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: +---[RSA 3072]----+ 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: | .+o..o. | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |o .*oo | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |+=++% | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |+=*=.B . | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |.++.+ . S . E | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |.. *. = . . | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |.o. o.. o o | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: |. ....oo . | 2024-04-26T20:59:31.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: | . o+. | 2024-04-26T20:59:31.987 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: +----[SHA256]-----+ 2024-04-26T20:59:32.428 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.273+0000 7f136fa00700 1 Processor -- start 2024-04-26T20:59:32.428 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.274+0000 7f136fa00700 1 -- start start 2024-04-26T20:59:32.428 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.274+0000 7f136fa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:32.428 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.274+0000 7f136fa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f136807ceb0 con 0x7f1368109920 2024-04-26T20:59:32.428 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.275+0000 7f136d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:32.429 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.275+0000 7f136d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58814/0 (socket says 172.21.15.27:58814) 2024-04-26T20:59:32.429 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.275+0000 7f136d000700 1 -- 172.21.15.27:0/541268542 learned_addr learned my addr 172.21.15.27:0/541268542 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:32.429 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.275+0000 7f136d000700 1 -- 172.21.15.27:0/541268542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136810c250 con 0x7f1368109920 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.276+0000 7f136d000700 1 --2- 172.21.15.27:0/541268542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098a0 tx=0x7f135c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6b4bcaded09ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.276+0000 7f1367400700 1 -- 172.21.15.27:0/541268542 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f135c004160 con 0x7f1368109920 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.276+0000 7f1367400700 1 -- 172.21.15.27:0/541268542 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f135c004300 con 0x7f1368109920 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.276+0000 7f1367400700 1 -- 172.21.15.27:0/541268542 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f135c0119e0 con 0x7f1368109920 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 -- 172.21.15.27:0/541268542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 msgr2=0x7f136810bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 --2- 172.21.15.27:0/541268542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098a0 tx=0x7f135c009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:32.431 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 -- 172.21.15.27:0/541268542 shutdown_connections 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 --2- 172.21.15.27:0/541268542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136810bd10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 -- 172.21.15.27:0/541268542 >> 172.21.15.27:0/541268542 conn(0x7f1368103290 msgr2=0x7f13681056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 -- 172.21.15.27:0/541268542 shutdown_connections 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.277+0000 7f136fa00700 1 -- 172.21.15.27:0/541268542 wait complete. 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.278+0000 7f136fa00700 1 Processor -- start 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.278+0000 7f136fa00700 1 -- start start 2024-04-26T20:59:32.432 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136fa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:32.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136fa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f136810d210 con 0x7f1368109920 2024-04-26T20:59:32.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:32.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58822/0 (socket says 172.21.15.27:58822) 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136d000700 1 -- 172.21.15.27:0/1156020392 learned_addr learned my addr 172.21.15.27:0/1156020392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.279+0000 7f136d000700 1 -- 172.21.15.27:0/1156020392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136819d290 con 0x7f1368109920 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f136d000700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f135c0119e0 tx=0x7f135c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f135c003bf0 con 0x7f1368109920 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f135c01a8a0 con 0x7f1368109920 2024-04-26T20:59:32.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f135c0248c0 con 0x7f1368109920 2024-04-26T20:59:32.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136819d520 con 0x7f1368109920 2024-04-26T20:59:32.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.280+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681a0210 con 0x7f1368109920 2024-04-26T20:59:32.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.281+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f135c022020 con 0x7f1368109920 2024-04-26T20:59:32.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.281+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368064a80 con 0x7f1368109920 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.282+0000 7f1365600700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 0x7f135003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.282+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f135c04dab0 con 0x7f1368109920 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.282+0000 7f1367e00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 0x7f135003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.283+0000 7f1367e00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 0x7f135003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1358006fd0 tx=0x7f1358006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.288+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f135c04d400 con 0x7f1368109920 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.520+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1368104480 con 0x7f1350039560 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.692+0000 7f1365600700 1 -- 172.21.15.27:0/1156020392 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1368104480 con 0x7f1350039560 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 msgr2=0x7f135003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:32.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 0x7f135003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1358006fd0 tx=0x7f1358006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 msgr2=0x7f136819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f135c0119e0 tx=0x7f135c0046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 shutdown_connections 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f1350039560 0x7f135003ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 --2- 172.21.15.27:0/1156020392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1368109920 0x7f136819cd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.695+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 >> 172.21.15.27:0/1156020392 conn(0x7f1368103290 msgr2=0x7f1368103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.696+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 shutdown_connections 2024-04-26T20:59:32.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:31.696+0000 7f136fa00700 1 -- 172.21.15.27:0/1156020392 wait complete. 2024-04-26T20:59:32.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:32 smithi027 ceph-mon[28085]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:32.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:32 smithi027 ceph-mon[28085]: Generating ssh key... 2024-04-26T20:59:32.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:32 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:32.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:32 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:33.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:33 smithi027 ceph-mon[28085]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:34.049 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.031+0000 7fb4c2800700 1 Processor -- start 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.032+0000 7fb4c2800700 1 -- start start 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.032+0000 7fb4c2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.032+0000 7fb4c2800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc07d8f0 con 0x7fb4bc07eec0 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.033+0000 7fb4bb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.033+0000 7fb4bb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58830/0 (socket says 172.21.15.27:58830) 2024-04-26T20:59:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.033+0000 7fb4bb400700 1 -- 172.21.15.27:0/537758899 learned_addr learned my addr 172.21.15.27:0/537758899 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.033+0000 7fb4bb400700 1 -- 172.21.15.27:0/537758899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc07da70 con 0x7fb4bc07eec0 2024-04-26T20:59:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.033+0000 7fb4bb400700 1 --2- 172.21.15.27:0/537758899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098a0 tx=0x7fb4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ff22aeb8831b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:34.057 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4ba000700 1 -- 172.21.15.27:0/537758899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ac004160 con 0x7fb4bc07eec0 2024-04-26T20:59:34.057 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4ba000700 1 -- 172.21.15.27:0/537758899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb4ac004300 con 0x7fb4bc07eec0 2024-04-26T20:59:34.057 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4ba000700 1 -- 172.21.15.27:0/537758899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ac0119e0 con 0x7fb4bc07eec0 2024-04-26T20:59:34.057 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4c2800700 1 -- 172.21.15.27:0/537758899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 msgr2=0x7fb4bc07d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4c2800700 1 --2- 172.21.15.27:0/537758899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098a0 tx=0x7fb4ac009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4c2800700 1 -- 172.21.15.27:0/537758899 shutdown_connections 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4c2800700 1 --2- 172.21.15.27:0/537758899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc07d320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.034+0000 7fb4c2800700 1 -- 172.21.15.27:0/537758899 >> 172.21.15.27:0/537758899 conn(0x7fb4bc104370 msgr2=0x7fb4bc106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.035+0000 7fb4c2800700 1 -- 172.21.15.27:0/537758899 shutdown_connections 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.035+0000 7fb4c2800700 1 -- 172.21.15.27:0/537758899 wait complete. 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4c2800700 1 Processor -- start 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4c2800700 1 -- start start 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4c2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4c2800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc10f2f0 con 0x7fb4bc07eec0 2024-04-26T20:59:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4bb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4bb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58846/0 (socket says 172.21.15.27:58846) 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.036+0000 7fb4bb400700 1 -- 172.21.15.27:0/2833115364 learned_addr learned my addr 172.21.15.27:0/2833115364 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4bb400700 1 -- 172.21.15.27:0/2833115364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc19f380 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4bb400700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006b20 tx=0x7fb4ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ac003a60 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb4ac003c00 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ac024780 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.037+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc19f610 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.038+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc1a2300 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.039+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc064a80 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.039+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb4ac022020 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.040+0000 7fb4b3e00700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 0x7fb4a003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.040+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4ac04db20 con 0x7fb4bc07eec0 2024-04-26T20:59:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.040+0000 7fb4baa00700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 0x7fb4a003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.041+0000 7fb4baa00700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 0x7fb4a003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006fd0 tx=0x7fb4a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.045+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb4ac00f2e0 con 0x7fb4bc07eec0 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.280+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb4bc1055e0 con 0x7fb4a0039600 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.281+0000 7fb4b3e00700 1 -- 172.21.15.27:0/2833115364 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb4bc1055e0 con 0x7fb4a0039600 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.284+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 msgr2=0x7fb4a003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.284+0000 7fb4c2800700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 0x7fb4a003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006fd0 tx=0x7fb4a8006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.284+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 msgr2=0x7fb4bc19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.284+0000 7fb4c2800700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006b20 tx=0x7fb4ac011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 shutdown_connections 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fb4a0039600 0x7fb4a003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 --2- 172.21.15.27:0/2833115364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4bc07eec0 0x7fb4bc19ee40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 >> 172.21.15.27:0/2833115364 conn(0x7fb4bc104370 msgr2=0x7fb4bc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 shutdown_connections 2024-04-26T20:59:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:33.285+0000 7fb4c2800700 1 -- 172.21.15.27:0/2833115364 wait complete. 2024-04-26T20:59:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T20:59:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T20:59:34.075 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-04-26T20:59:35.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:35 smithi027 ceph-mon[28085]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:36.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:36 smithi027 ceph-mon[28085]: Deploying cephadm binary to smithi027 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.723+0000 7f6464200700 1 Processor -- start 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.723+0000 7f6464200700 1 -- start start 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.723+0000 7f6464200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.724+0000 7f6464200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f645c112670 con 0x7f645c10fcb0 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.724+0000 7f6461800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.724+0000 7f6461800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58854/0 (socket says 172.21.15.27:58854) 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.724+0000 7f6461800700 1 -- 172.21.15.27:0/655466962 learned_addr learned my addr 172.21.15.27:0/655466962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.724+0000 7f6461800700 1 -- 172.21.15.27:0/655466962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1127f0 con 0x7f645c10fcb0 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.725+0000 7f6461800700 1 --2- 172.21.15.27:0/655466962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64500098a0 tx=0x7f6450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0b7694b701a94a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.725+0000 7f645be00700 1 -- 172.21.15.27:0/655466962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6450004160 con 0x7f645c10fcb0 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.725+0000 7f645be00700 1 -- 172.21.15.27:0/655466962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6450004300 con 0x7f645c10fcb0 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.725+0000 7f645be00700 1 -- 172.21.15.27:0/655466962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64500119e0 con 0x7f645c10fcb0 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 -- 172.21.15.27:0/655466962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 msgr2=0x7f645c1120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 --2- 172.21.15.27:0/655466962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64500098a0 tx=0x7f6450009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 -- 172.21.15.27:0/655466962 shutdown_connections 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 --2- 172.21.15.27:0/655466962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1120a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 -- 172.21.15.27:0/655466962 >> 172.21.15.27:0/655466962 conn(0x7f645c07e3d0 msgr2=0x7f645c0807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 -- 172.21.15.27:0/655466962 shutdown_connections 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.726+0000 7f6464200700 1 -- 172.21.15.27:0/655466962 wait complete. 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.727+0000 7f6464200700 1 Processor -- start 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.727+0000 7f6464200700 1 -- start start 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6464200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6464200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f645c113810 con 0x7f645c10fcb0 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6461800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:42.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6461800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58864/0 (socket says 172.21.15.27:58864) 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6461800700 1 -- 172.21.15.27:0/975293408 learned_addr learned my addr 172.21.15.27:0/975293408 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.728+0000 7f6461800700 1 -- 172.21.15.27:0/975293408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1a3890 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f6461800700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f64500116a0 tx=0x7f64500046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6450003b70 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f645001a8a0 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6450024890 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1a3b20 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.729+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1a6810 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.730+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6450022020 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.730+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c064a80 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.731+0000 7f645a000700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 0x7f644403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.731+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f645004db50 con 0x7f645c10fcb0 2024-04-26T20:59:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.731+0000 7f6460e00700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 0x7f644403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.732+0000 7f6460e00700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 0x7f644403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6448006fd0 tx=0x7f6448006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.737+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f645000f2e0 con 0x7f645c10fcb0 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:34.972+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) -- 0x7f645c07f5b0 con 0x7f64440395b0 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.484+0000 7f645a000700 1 -- 172.21.15.27:0/975293408 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f645c07f5b0 con 0x7f64440395b0 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 msgr2=0x7f644403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 0x7f644403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6448006fd0 tx=0x7f6448006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 msgr2=0x7f645c1a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f64500116a0 tx=0x7f64500046a0 comp rx=0 tx=0).stop 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 shutdown_connections 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f64440395b0 0x7f644403ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 --2- 172.21.15.27:0/975293408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f645c10fcb0 0x7f645c1a3350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.489+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 >> 172.21.15.27:0/975293408 conn(0x7f645c07e3d0 msgr2=0x7f645c07eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.490+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 shutdown_connections 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:41.490+0000 7f6464200700 1 -- 172.21.15.27:0/975293408 wait complete. 2024-04-26T20:59:42.331 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-04-26T20:59:42.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:42 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:42.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:42 smithi027 ceph-mon[28085]: Added host smithi027 2024-04-26T20:59:42.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:42 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:44 smithi027 ceph-mon[28085]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:44 smithi027 ceph-mon[28085]: Saving service mon spec with placement count:5 2024-04-26T20:59:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:44 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.003+0000 7f6d99600700 1 Processor -- start 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.003+0000 7f6d99600700 1 -- start start 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.004+0000 7f6d99600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.004+0000 7f6d99600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9407d8d0 con 0x7f6d9407eea0 2024-04-26T20:59:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.005+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.005+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48336/0 (socket says 172.21.15.27:48336) 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.005+0000 7f6d92a00700 1 -- 172.21.15.27:0/3784204305 learned_addr learned my addr 172.21.15.27:0/3784204305 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.005+0000 7f6d92a00700 1 -- 172.21.15.27:0/3784204305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9407da50 con 0x7f6d9407eea0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d92a00700 1 --2- 172.21.15.27:0/3784204305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098a0 tx=0x7f6d84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c198377eda729a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d91600700 1 -- 172.21.15.27:0/3784204305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d84004160 con 0x7f6d9407eea0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d91600700 1 -- 172.21.15.27:0/3784204305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6d84004300 con 0x7f6d9407eea0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d91600700 1 -- 172.21.15.27:0/3784204305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d840119e0 con 0x7f6d9407eea0 2024-04-26T20:59:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d99600700 1 -- 172.21.15.27:0/3784204305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 msgr2=0x7f6d9407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.006+0000 7f6d99600700 1 --2- 172.21.15.27:0/3784204305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098a0 tx=0x7f6d84009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.007+0000 7f6d99600700 1 -- 172.21.15.27:0/3784204305 shutdown_connections 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.007+0000 7f6d99600700 1 --2- 172.21.15.27:0/3784204305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9407d300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.007+0000 7f6d99600700 1 -- 172.21.15.27:0/3784204305 >> 172.21.15.27:0/3784204305 conn(0x7f6d94104350 msgr2=0x7f6d94106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.007+0000 7f6d99600700 1 -- 172.21.15.27:0/3784204305 shutdown_connections 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.007+0000 7f6d99600700 1 -- 172.21.15.27:0/3784204305 wait complete. 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.008+0000 7f6d99600700 1 Processor -- start 2024-04-26T20:59:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.008+0000 7f6d99600700 1 -- start start 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.008+0000 7f6d99600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.008+0000 7f6d99600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9410f2c0 con 0x7f6d9407eea0 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.009+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.009+0000 7f6d92a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48338/0 (socket says 172.21.15.27:48338) 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.009+0000 7f6d92a00700 1 -- 172.21.15.27:0/2333271501 learned_addr learned my addr 172.21.15.27:0/2333271501 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.009+0000 7f6d92a00700 1 -- 172.21.15.27:0/2333271501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9419f380 con 0x7f6d9407eea0 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d92a00700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d84006b20 tx=0x7f6d84011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d84003a60 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6d84003c00 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9419f610 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d84024780 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.010+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941a2300 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.011+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d94064a80 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.011+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6d84022020 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.012+0000 7f6d8b400700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 0x7f6d7803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.012+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6d8404dbb0 con 0x7f6d9407eea0 2024-04-26T20:59:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.012+0000 7f6d92000700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 0x7f6d7803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.013+0000 7f6d92000700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 0x7f6d7803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006fd0 tx=0x7f6d80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.018+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6d8401ac50 con 0x7f6d9407eea0 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.302+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6d94105550 con 0x7f6d78039600 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.311+0000 7f6d8b400700 1 -- 172.21.15.27:0/2333271501 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6d94105550 con 0x7f6d78039600 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 msgr2=0x7f6d7803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 0x7f6d7803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006fd0 tx=0x7f6d80006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 msgr2=0x7f6d9419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d84006b20 tx=0x7f6d84011a60 comp rx=0 tx=0).stop 2024-04-26T20:59:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 shutdown_connections 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f6d78039600 0x7f6d7803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 --2- 172.21.15.27:0/2333271501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9407eea0 0x7f6d9419ee40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 >> 172.21.15.27:0/2333271501 conn(0x7f6d94104350 msgr2=0x7f6d94104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 shutdown_connections 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:43.314+0000 7f6d99600700 1 -- 172.21.15.27:0/2333271501 wait complete. 2024-04-26T20:59:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-04-26T20:59:46.685 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:46.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:59:46.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T20:59:46.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: Saving service mgr spec with placement count:2 2024-04-26T20:59:46.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:46.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:46 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:46.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.731+0000 7f2c7d200700 1 Processor -- start 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.731+0000 7f2c7d200700 1 -- start start 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.731+0000 7f2c7d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.731+0000 7f2c7d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78109780 con 0x7f2c78108d90 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.732+0000 7f2c75e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.732+0000 7f2c75e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48346/0 (socket says 172.21.15.27:48346) 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.732+0000 7f2c75e00700 1 -- 172.21.15.27:0/1486441557 learned_addr learned my addr 172.21.15.27:0/1486441557 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.732+0000 7f2c75e00700 1 -- 172.21.15.27:0/1486441557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c78109f90 con 0x7f2c78108d90 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.733+0000 7f2c75e00700 1 --2- 172.21.15.27:0/1486441557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098a0 tx=0x7f2c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2defaded6d498192 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.733+0000 7f2c74a00700 1 -- 172.21.15.27:0/1486441557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c68004160 con 0x7f2c78108d90 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.733+0000 7f2c74a00700 1 -- 172.21.15.27:0/1486441557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c68004300 con 0x7f2c78108d90 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.733+0000 7f2c74a00700 1 -- 172.21.15.27:0/1486441557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c68011960 con 0x7f2c78108d90 2024-04-26T20:59:46.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 -- 172.21.15.27:0/1486441557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 msgr2=0x7f2c781091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 --2- 172.21.15.27:0/1486441557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098a0 tx=0x7f2c68009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 -- 172.21.15.27:0/1486441557 shutdown_connections 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 --2- 172.21.15.27:0/1486441557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c781091b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 -- 172.21.15.27:0/1486441557 >> 172.21.15.27:0/1486441557 conn(0x7f2c78104350 msgr2=0x7f2c78106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 -- 172.21.15.27:0/1486441557 shutdown_connections 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.734+0000 7f2c7d200700 1 -- 172.21.15.27:0/1486441557 wait complete. 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.735+0000 7f2c7d200700 1 Processor -- start 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c7d200700 1 -- start start 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c7d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c7d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7810d1c0 con 0x7f2c78108d90 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c75e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:46.704 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c75e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48348/0 (socket says 172.21.15.27:48348) 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.736+0000 7f2c75e00700 1 -- 172.21.15.27:0/2292309775 learned_addr learned my addr 172.21.15.27:0/2292309775 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c75e00700 1 -- 172.21.15.27:0/2292309775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7807f090 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c75e00700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c68011670 tx=0x7f2c68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c6801f070 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c68003b20 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c68019580 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.737+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7807d280 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.738+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7807d7c0 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.739+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7810d340 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.739+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2c68022070 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.740+0000 7f2c6ea00700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 0x7f2c5c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.740+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c6804da30 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.740+0000 7f2c75400700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 0x7f2c5c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.741+0000 7f2c75400700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 0x7f2c5c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c64006fd0 tx=0x7f2c64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.745+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2c6801b440 con 0x7f2c78108d90 2024-04-26T20:59:46.705 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:45.993+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2c7807e040 con 0x7f2c5c039600 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.002+0000 7f2c6ea00700 1 -- 172.21.15.27:0/2292309775 <== mgr.14120 v2:172.21.15.27:6800/128242469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2c7807e040 con 0x7f2c5c039600 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 msgr2=0x7f2c5c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 0x7f2c5c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c64006fd0 tx=0x7f2c64006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 msgr2=0x7f2c7807eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c68011670 tx=0x7f2c68011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 shutdown_connections 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f2c5c039600 0x7f2c5c03bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 --2- 172.21.15.27:0/2292309775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c78108d90 0x7f2c7807eb50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.005+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 >> 172.21.15.27:0/2292309775 conn(0x7f2c78104350 msgr2=0x7f2c78104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.006+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 shutdown_connections 2024-04-26T20:59:46.706 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:46.006+0000 7f2c7d200700 1 -- 172.21.15.27:0/2292309775 wait complete. 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.295+0000 7f096b800700 1 Processor -- start 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.295+0000 7f096b800700 1 -- start start 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.295+0000 7f096b800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.296+0000 7f096b800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f096407d8f0 con 0x7f096407eec0 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.296+0000 7f0968e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.296+0000 7f0968e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48360/0 (socket says 172.21.15.27:48360) 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.296+0000 7f0968e00700 1 -- 172.21.15.27:0/3434394919 learned_addr learned my addr 172.21.15.27:0/3434394919 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:48.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.296+0000 7f0968e00700 1 -- 172.21.15.27:0/3434394919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096407da70 con 0x7f096407eec0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.297+0000 7f0968e00700 1 --2- 172.21.15.27:0/3434394919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2316f509640716e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.297+0000 7f0963400700 1 -- 172.21.15.27:0/3434394919 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958004160 con 0x7f096407eec0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.297+0000 7f0963400700 1 -- 172.21.15.27:0/3434394919 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0958004300 con 0x7f096407eec0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.297+0000 7f0963400700 1 -- 172.21.15.27:0/3434394919 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958011960 con 0x7f096407eec0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 -- 172.21.15.27:0/3434394919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 msgr2=0x7f096407d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 --2- 172.21.15.27:0/3434394919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 -- 172.21.15.27:0/3434394919 shutdown_connections 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 --2- 172.21.15.27:0/3434394919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f096407d320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 -- 172.21.15.27:0/3434394919 >> 172.21.15.27:0/3434394919 conn(0x7f09641043b0 msgr2=0x7f09641067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 -- 172.21.15.27:0/3434394919 shutdown_connections 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.298+0000 7f096b800700 1 -- 172.21.15.27:0/3434394919 wait complete. 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.299+0000 7f096b800700 1 Processor -- start 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.299+0000 7f096b800700 1 -- start start 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f096b800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f096b800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f096410af70 con 0x7f096407eec0 2024-04-26T20:59:48.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f0968e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f0968e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48376/0 (socket says 172.21.15.27:48376) 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f0968e00700 1 -- 172.21.15.27:0/2729746922 learned_addr learned my addr 172.21.15.27:0/2729746922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.300+0000 7f0968e00700 1 -- 172.21.15.27:0/2729746922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09641a37e0 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f0968e00700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0958009520 tx=0x7f0958011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958003aa0 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f095801b8d0 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09641a3a70 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958023790 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.301+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09641a6760 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.303+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0964064a80 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.303+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0958023a50 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.303+0000 7f0961600700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 0x7f094c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.303+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f095804ded0 con 0x7f096407eec0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.304+0000 7f0963e00700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 0x7f094c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.304+0000 7f0963e00700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 0x7f094c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.308+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f095801f030 con 0x7f096407eec0 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.569+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f096402e700 con 0x7f096407eec0 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.573+0000 7f0961600700 1 -- 172.21.15.27:0/2729746922 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0958023df0 con 0x7f096407eec0 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 msgr2=0x7f094c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 0x7f094c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 msgr2=0x7f09641a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0958009520 tx=0x7f0958011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 shutdown_connections 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f094c0395b0 0x7f094c03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 --2- 172.21.15.27:0/2729746922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096407eec0 0x7f09641a32a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 >> 172.21.15.27:0/2729746922 conn(0x7f09641043b0 msgr2=0x7f09641067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:48.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.576+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 shutdown_connections 2024-04-26T20:59:48.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:47.577+0000 7f096b800700 1 -- 172.21.15.27:0/2729746922 wait complete. 2024-04-26T20:59:48.628 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:48 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2729746922' entity='client.admin' 2024-04-26T20:59:48.628 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:48 smithi027 ceph-mon[28085]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:59:50.451 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:50 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:50.451 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:50 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/429497309' entity='client.admin' 2024-04-26T20:59:50.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.478+0000 7fc774000700 1 Processor -- start 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.478+0000 7fc774000700 1 -- start start 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.478+0000 7fc774000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.478+0000 7fc774000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c07d8b0 con 0x7fc76c07ee80 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.479+0000 7fc771600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.479+0000 7fc771600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53022/0 (socket says 172.21.15.27:53022) 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.479+0000 7fc771600700 1 -- 172.21.15.27:0/2438605487 learned_addr learned my addr 172.21.15.27:0/2438605487 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.479+0000 7fc771600700 1 -- 172.21.15.27:0/2438605487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c07da30 con 0x7fc76c07ee80 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.479+0000 7fc771600700 1 --2- 172.21.15.27:0/2438605487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098a0 tx=0x7fc760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ecb91ab9291a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:50.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.480+0000 7fc76be00700 1 -- 172.21.15.27:0/2438605487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc760004160 con 0x7fc76c07ee80 2024-04-26T20:59:50.479 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.480+0000 7fc76be00700 1 -- 172.21.15.27:0/2438605487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc760004300 con 0x7fc76c07ee80 2024-04-26T20:59:50.479 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.480+0000 7fc76be00700 1 -- 172.21.15.27:0/2438605487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc760011960 con 0x7fc76c07ee80 2024-04-26T20:59:50.479 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 -- 172.21.15.27:0/2438605487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 msgr2=0x7fc76c07d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:50.479 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 --2- 172.21.15.27:0/2438605487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098a0 tx=0x7fc760009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:50.479 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 -- 172.21.15.27:0/2438605487 shutdown_connections 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 --2- 172.21.15.27:0/2438605487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c07d2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 -- 172.21.15.27:0/2438605487 >> 172.21.15.27:0/2438605487 conn(0x7fc76c104350 msgr2=0x7fc76c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 -- 172.21.15.27:0/2438605487 shutdown_connections 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.481+0000 7fc774000700 1 -- 172.21.15.27:0/2438605487 wait complete. 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.482+0000 7fc774000700 1 Processor -- start 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.482+0000 7fc774000700 1 -- start start 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc774000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc774000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c10f330 con 0x7fc76c07ee80 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc771600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:50.483 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc771600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53026/0 (socket says 172.21.15.27:53026) 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc771600700 1 -- 172.21.15.27:0/429497309 learned_addr learned my addr 172.21.15.27:0/429497309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.483+0000 7fc771600700 1 -- 172.21.15.27:0/429497309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c19f450 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc771600700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc760011670 tx=0x7fc760011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc760003aa0 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc76001b8d0 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c19f6e0 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7600197f0 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.484+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a23d0 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.485+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc760019990 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.486+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c064a80 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.486+0000 7fc76a000700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 0x7fc75403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.486+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc76004df80 con 0x7fc76c07ee80 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.486+0000 7fc770c00700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 0x7fc75403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:50.484 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.487+0000 7fc770c00700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 0x7fc75403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc758006fd0 tx=0x7fc758006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.492+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc76001f030 con 0x7fc76c07ee80 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.727+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc76c1a29f0 con 0x7fc76c07ee80 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.733+0000 7fc76a000700 1 -- 172.21.15.27:0/429497309 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc760016df0 con 0x7fc76c07ee80 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.736+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 msgr2=0x7fc75403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.736+0000 7fc774000700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 0x7fc75403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc758006fd0 tx=0x7fc758006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.736+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 msgr2=0x7fc76c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.736+0000 7fc774000700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc760011670 tx=0x7fc760011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.737+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 shutdown_connections 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.737+0000 7fc774000700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7fc7540395b0 0x7fc75403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.737+0000 7fc774000700 1 --2- 172.21.15.27:0/429497309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc76c07ee80 0x7fc76c19ef10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.737+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 >> 172.21.15.27:0/429497309 conn(0x7fc76c104350 msgr2=0x7fc76c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.737+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 shutdown_connections 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:49.738+0000 7fc774000700 1 -- 172.21.15.27:0/429497309 wait complete. 2024-04-26T20:59:50.485 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-04-26T20:59:51.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:51 smithi027 ceph-mon[28085]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:59:52.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-mon[28085]: from='mgr.14120 172.21.15.27:0/2955618541' entity='mgr.x' 2024-04-26T20:59:52.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2886180712' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T20:59:52.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-mon[28085]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T20:59:52.709 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setuser ceph since I am not root 2024-04-26T20:59:52.709 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setgroup ceph since I am not root 2024-04-26T20:59:52.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:52.807+0000 7f07b5393200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:59:53.475 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:53.069+0000 7f07b5393200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:53.475 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:53.209+0000 7f07b5393200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:59:53.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2886180712' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T20:59:53.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-mon[28085]: mgrmap e9: x(active, since 26s) 2024-04-26T20:59:53.735 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:53.625+0000 7f07b5393200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:59:53.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.310+0000 7f41bb000700 1 Processor -- start 2024-04-26T20:59:53.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.311+0000 7f41bb000700 1 -- start start 2024-04-26T20:59:53.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.311+0000 7f41bb000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:53.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.311+0000 7f41bb000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41097b0 con 0x7f41b4108dc0 2024-04-26T20:59:53.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.311+0000 7f41b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.312+0000 7f41b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53042/0 (socket says 172.21.15.27:53042) 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.312+0000 7f41b3e00700 1 -- 172.21.15.27:0/2468215900 learned_addr learned my addr 172.21.15.27:0/2468215900 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.312+0000 7f41b3e00700 1 -- 172.21.15.27:0/2468215900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4109fc0 con 0x7f41b4108dc0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.312+0000 7f41b3e00700 1 --2- 172.21.15.27:0/2468215900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41a40098a0 tx=0x7f41a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b973b80dd5f67e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.313+0000 7f41b2a00700 1 -- 172.21.15.27:0/2468215900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41a4004160 con 0x7f41b4108dc0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.313+0000 7f41b2a00700 1 -- 172.21.15.27:0/2468215900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41a4004300 con 0x7f41b4108dc0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.313+0000 7f41b2a00700 1 -- 172.21.15.27:0/2468215900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41a4011960 con 0x7f41b4108dc0 2024-04-26T20:59:53.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.313+0000 7f41bb000700 1 -- 172.21.15.27:0/2468215900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 msgr2=0x7f41b41091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.313+0000 7f41bb000700 1 --2- 172.21.15.27:0/2468215900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41a40098a0 tx=0x7f41a4009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.314+0000 7f41bb000700 1 -- 172.21.15.27:0/2468215900 shutdown_connections 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.314+0000 7f41bb000700 1 --2- 172.21.15.27:0/2468215900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b41091e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.314+0000 7f41bb000700 1 -- 172.21.15.27:0/2468215900 >> 172.21.15.27:0/2468215900 conn(0x7f41b41043a0 msgr2=0x7f41b41067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.314+0000 7f41bb000700 1 -- 172.21.15.27:0/2468215900 shutdown_connections 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.314+0000 7f41bb000700 1 -- 172.21.15.27:0/2468215900 wait complete. 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.315+0000 7f41bb000700 1 Processor -- start 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.315+0000 7f41bb000700 1 -- start start 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.315+0000 7f41bb000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.315+0000 7f41bb000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410d1f0 con 0x7f41b4108dc0 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.316+0000 7f41b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.316+0000 7f41b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53046/0 (socket says 172.21.15.27:53046) 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.316+0000 7f41b3e00700 1 -- 172.21.15.27:0/2886180712 learned_addr learned my addr 172.21.15.27:0/2886180712 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.316+0000 7f41b3e00700 1 -- 172.21.15.27:0/2886180712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b419f3a0 con 0x7f41b4108dc0 2024-04-26T20:59:53.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.316+0000 7f41b3e00700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41a4011670 tx=0x7f41a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.317+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41a401f070 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.317+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41a4003b20 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.317+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b419f630 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.317+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41a4019580 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.317+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41a2320 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.318+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b410d370 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.319+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f41a4022070 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.319+0000 7f41b0c00700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 0x7f419803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.319+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41a404db50 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.320+0000 7f41b3400700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 0x7f419803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.320+0000 7f41b3400700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 0x7f419803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f41a0006fd0 tx=0x7f41a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.325+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f41a404d4a0 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:51.629+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f41b41a24f0 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.474+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f41a401b440 con 0x7f41b4108dc0 2024-04-26T20:59:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.477+0000 7f41b0c00700 1 -- 172.21.15.27:0/2886180712 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f41a401ba80 con 0x7f41b4108dc0 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 msgr2=0x7f419803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 0x7f419803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f41a0006fd0 tx=0x7f41a0006e40 comp rx=0 tx=0).stop 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 msgr2=0x7f41b419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41a4011670 tx=0x7f41a4011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 shutdown_connections 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f41980395b0 0x7f419803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 --2- 172.21.15.27:0/2886180712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41b4108dc0 0x7f41b419ee60 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41a4011670 tx=0x7f41a4011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 >> 172.21.15.27:0/2886180712 conn(0x7f41b41043a0 msgr2=0x7f41b4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.479+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 shutdown_connections 2024-04-26T20:59:53.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:52.480+0000 7f41bb000700 1 -- 172.21.15.27:0/2886180712 wait complete. 2024-04-26T20:59:54.154 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:53.743+0000 7f07b5393200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:59:55.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 20:59:55 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1004461450' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T20:59:55.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:55.387+0000 7f07b5393200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:59:55.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:55.624+0000 7f07b5393200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:59:56.095 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:55.847+0000 7f07b5393200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:59:56.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T20:59:56.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T20:59:56.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T20:59:56.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T20:59:56.107 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T20:59:56.107 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.799+0000 7f7da4200700 1 Processor -- start 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.799+0000 7f7da4200700 1 -- start start 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.800+0000 7f7da4200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.800+0000 7f7da4200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c07d8d0 con 0x7f7d9c07eea0 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.800+0000 7f7da1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.800+0000 7f7da1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53078/0 (socket says 172.21.15.27:53078) 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.800+0000 7f7da1800700 1 -- 172.21.15.27:0/2880927430 learned_addr learned my addr 172.21.15.27:0/2880927430 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.801+0000 7f7da1800700 1 -- 172.21.15.27:0/2880927430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c07da50 con 0x7f7d9c07eea0 2024-04-26T20:59:56.110 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.801+0000 7f7da1800700 1 --2- 172.21.15.27:0/2880927430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d900098a0 tx=0x7f7d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fae4f6f3b0eb92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.801+0000 7f7d9be00700 1 -- 172.21.15.27:0/2880927430 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90004160 con 0x7f7d9c07eea0 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.801+0000 7f7d9be00700 1 -- 172.21.15.27:0/2880927430 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d90004300 con 0x7f7d9c07eea0 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.801+0000 7f7d9be00700 1 -- 172.21.15.27:0/2880927430 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90011960 con 0x7f7d9c07eea0 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 -- 172.21.15.27:0/2880927430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 msgr2=0x7f7d9c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 --2- 172.21.15.27:0/2880927430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d900098a0 tx=0x7f7d90009bb0 comp rx=0 tx=0).stop 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 -- 172.21.15.27:0/2880927430 shutdown_connections 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 --2- 172.21.15.27:0/2880927430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c07d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 -- 172.21.15.27:0/2880927430 >> 172.21.15.27:0/2880927430 conn(0x7f7d9c104350 msgr2=0x7f7d9c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 -- 172.21.15.27:0/2880927430 shutdown_connections 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.802+0000 7f7da4200700 1 -- 172.21.15.27:0/2880927430 wait complete. 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.803+0000 7f7da4200700 1 Processor -- start 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da4200700 1 -- start start 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da4200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da4200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10f310 con 0x7f7d9c07eea0 2024-04-26T20:59:56.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:59:56.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53094/0 (socket says 172.21.15.27:53094) 2024-04-26T20:59:56.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.804+0000 7f7da1800700 1 -- 172.21.15.27:0/1004461450 learned_addr learned my addr 172.21.15.27:0/1004461450 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T20:59:56.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7da1800700 1 -- 172.21.15.27:0/1004461450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c19f3a0 con 0x7f7d9c07eea0 2024-04-26T20:59:56.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7da1800700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d90011670 tx=0x7f7d90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d9001f070 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d90003b20 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c19f630 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90019580 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.805+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a2320 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.806+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7d90022070 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.807+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c064a80 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.807+0000 7f7d9a000700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 0x7f7d8403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.807+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7d9004dcf0 con 0x7f7d9c07eea0 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.807+0000 7f7da0e00700 1 -- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 msgr2=0x7f7d8403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T20:59:56.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.807+0000 7f7da0e00700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 0x7f7d8403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:54.813+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7d90022360 con 0x7f7d9c07eea0 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.008+0000 7f7da0e00700 1 -- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 msgr2=0x7f7d8403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.008+0000 7f7da0e00700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 0x7f7d8403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.101+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7d9c1a24f0 con 0x7f7d9c07eea0 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.102+0000 7f7d9a000700 1 -- 172.21.15.27:0/1004461450 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7d90019d60 con 0x7f7d9c07eea0 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 msgr2=0x7f7d8403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 0x7f7d8403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 msgr2=0x7f7d9c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d90011670 tx=0x7f7d90011a30 comp rx=0 tx=0).stop 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 shutdown_connections 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f7d84039660 0x7f7d8403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 --2- 172.21.15.27:0/1004461450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d9c07eea0 0x7f7d9c19ee60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.106+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 >> 172.21.15.27:0/1004461450 conn(0x7f7d9c104350 msgr2=0x7f7d9c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.107+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 shutdown_connections 2024-04-26T20:59:56.116 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:55.107+0000 7f7da4200700 1 -- 172.21.15.27:0/1004461450 wait complete. 2024-04-26T20:59:56.117 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-04-26T20:59:56.117 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2024-04-26T20:59:56.389 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:56.256+0000 7f07b5393200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:59:56.389 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:56.387+0000 7f07b5393200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:59:56.924 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:56.657+0000 7f07b5393200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:59:57.234 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:56.922+0000 7f07b5393200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:59:58.128 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:57.757+0000 7f07b5393200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:59:58.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:58.349+0000 7f07b5393200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:59:58.769 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:58.487+0000 7f07b5393200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:59:59.053 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:58.767+0000 7f07b5393200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:59:59.730 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:59.455+0000 7f07b5393200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:59:59.730 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:59.584+0000 7f07b5393200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:59:59.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 20:59:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T20:59:59.919+0000 7f07b5393200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:00:00.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:00.104+0000 7f07b5393200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:00:00.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:00.236+0000 7f07b5393200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:00:00.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:00.394+0000 7f07b5393200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:00:01.234 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:00.777+0000 7f07b5393200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:00:01.868 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:01.569+0000 7f07b5393200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:00:01.868 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:00:01.866+0000 7f07b5393200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:00:02.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: Active manager daemon x restarted 2024-04-26T21:00:02.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: Activating manager daemon x 2024-04-26T21:00:02.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: mgrmap e10: x(active, starting, since 0.0112814s) 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:00:02.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:01 smithi027 ceph-mon[28085]: Manager daemon x is now available 2024-04-26T21:00:03.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:03.284 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:00:03.284 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:00:03.284 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:02 smithi027 ceph-mon[28085]: mgrmap e11: x(active, since 1.01007s) 2024-04-26T21:00:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:00:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:00:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.714+0000 7f11da400700 1 Processor -- start 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.715+0000 7f11da400700 1 -- start start 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.715+0000 7f11da400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.715+0000 7f11da400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41097c0 con 0x7f11d4108dd0 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.715+0000 7f11d3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.716+0000 7f11d3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53104/0 (socket says 172.21.15.27:53104) 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.716+0000 7f11d3400700 1 -- 172.21.15.27:0/3838796022 learned_addr learned my addr 172.21.15.27:0/3838796022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.716+0000 7f11d3400700 1 -- 172.21.15.27:0/3838796022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d4109fd0 con 0x7f11d4108dd0 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.716+0000 7f11d3400700 1 --2- 172.21.15.27:0/3838796022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f11c40098a0 tx=0x7f11c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397e63fdcbb49fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.717+0000 7f11d2000700 1 -- 172.21.15.27:0/3838796022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c4004160 con 0x7f11d4108dd0 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.717+0000 7f11d2000700 1 -- 172.21.15.27:0/3838796022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11c4004300 con 0x7f11d4108dd0 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.717+0000 7f11d2000700 1 -- 172.21.15.27:0/3838796022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c4011960 con 0x7f11d4108dd0 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.717+0000 7f11da400700 1 -- 172.21.15.27:0/3838796022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 msgr2=0x7f11d41091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.717+0000 7f11da400700 1 --2- 172.21.15.27:0/3838796022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f11c40098a0 tx=0x7f11c4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.718+0000 7f11da400700 1 -- 172.21.15.27:0/3838796022 shutdown_connections 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.718+0000 7f11da400700 1 --2- 172.21.15.27:0/3838796022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d41091f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.718+0000 7f11da400700 1 -- 172.21.15.27:0/3838796022 >> 172.21.15.27:0/3838796022 conn(0x7f11d41043b0 msgr2=0x7f11d41067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.718+0000 7f11da400700 1 -- 172.21.15.27:0/3838796022 shutdown_connections 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.718+0000 7f11da400700 1 -- 172.21.15.27:0/3838796022 wait complete. 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.719+0000 7f11da400700 1 Processor -- start 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.719+0000 7f11da400700 1 -- start start 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.720+0000 7f11da400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.720+0000 7f11da400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d200 con 0x7f11d4108dd0 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.720+0000 7f11d3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.720+0000 7f11d3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53118/0 (socket says 172.21.15.27:53118) 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.720+0000 7f11d3400700 1 -- 172.21.15.27:0/2936956366 learned_addr learned my addr 172.21.15.27:0/2936956366 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11d3400700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d407f090 con 0x7f11d4108dd0 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11d3400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011670 tx=0x7f11c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c401f070 con 0x7f11d4108dd0 2024-04-26T21:00:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11c4003b20 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d407d280 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.721+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c4019580 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.722+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d407d7c0 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f11c4022070 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11cbe00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f11b803c230 con 0x7f11b8039660 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11c404dab0 con 0x7f11d4108dd0 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11d2a00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.723+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.924+0000 7f11d2a00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:56.924+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:57.325+0000 7f11d2a00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:57.325+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:00:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:58.126+0000 7f11d2a00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:58.126+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:59.728+0000 7f11d2a00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/128242469 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T20:59:59.728+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:01.877+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f11c401b440 con 0x7f11d4108dd0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:01.877+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 msgr2=0x7f11b803bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:01.877+0000 7f11cbe00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.877+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f11c404f730 con 0x7f11d4108dd0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.877+0000 7f11cbe00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 0x7f11b803f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.877+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f11c400be90 con 0x7f11b803cff0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.878+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 0x7f11b803f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.881+0000 7f11d2a00700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 0x7f11b803f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11c0003a10 tx=0x7f11c0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.882+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f11c400be90 con 0x7f11b803cff0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.891+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f11d410d380 con 0x7f11b803cff0 2024-04-26T21:00:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11cbe00700 1 -- 172.21.15.27:0/2936956366 <== mgr.14152 v2:172.21.15.27:6800/3365757774 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f11d410d380 con 0x7f11b803cff0 2024-04-26T21:00:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 msgr2=0x7f11b803f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 0x7f11b803f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11c0003a10 tx=0x7f11c0009270 comp rx=0 tx=0).stop 2024-04-26T21:00:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 msgr2=0x7f11d407eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011670 tx=0x7f11c4011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 shutdown_connections 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f11b803cff0 0x7f11b803f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11d4108dd0 0x7f11d407eb50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 --2- 172.21.15.27:0/2936956366 >> [v2:172.21.15.27:6800/128242469,v1:172.21.15.27:6801/128242469] conn(0x7f11b8039660 0x7f11b803bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.892+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 >> 172.21.15.27:0/2936956366 conn(0x7f11d41043b0 msgr2=0x7f11d4104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.893+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 shutdown_connections 2024-04-26T21:00:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:02.893+0000 7f11da400700 1 -- 172.21.15.27:0/2936956366 wait complete. 2024-04-26T21:00:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2024-04-26T21:00:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:00:04.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:03 smithi027 ceph-mon[28085]: [26/Apr/2024:21:00:03] ENGINE Bus STARTING 2024-04-26T21:00:04.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:03 smithi027 ceph-mon[28085]: [26/Apr/2024:21:00:03] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T21:00:05.226 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:04 smithi027 ceph-mon[28085]: [26/Apr/2024:21:00:03] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T21:00:05.226 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:04 smithi027 ceph-mon[28085]: [26/Apr/2024:21:00:03] ENGINE Bus STARTED 2024-04-26T21:00:05.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:04 smithi027 ceph-mon[28085]: mgrmap e12: x(active, since 2s) 2024-04-26T21:00:05.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:05.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:00:05.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.837+0000 7f6bb4c00700 1 Processor -- start 2024-04-26T21:00:05.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.837+0000 7f6bb4c00700 1 -- start start 2024-04-26T21:00:05.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.838+0000 7f6bb4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.838+0000 7f6bb4c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb010b9b0 con 0x7f6bb010afc0 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.838+0000 7f6bade00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.839+0000 7f6bade00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33140/0 (socket says 172.21.15.27:33140) 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.839+0000 7f6bade00700 1 -- 172.21.15.27:0/3621342789 learned_addr learned my addr 172.21.15.27:0/3621342789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.839+0000 7f6bade00700 1 -- 172.21.15.27:0/3621342789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb010c1c0 con 0x7f6bb010afc0 2024-04-26T21:00:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.840+0000 7f6bade00700 1 --2- 172.21.15.27:0/3621342789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098a0 tx=0x7f6ba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19452991b26de28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.840+0000 7f6baca00700 1 -- 172.21.15.27:0/3621342789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba0004160 con 0x7f6bb010afc0 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.840+0000 7f6baca00700 1 -- 172.21.15.27:0/3621342789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba0004300 con 0x7f6bb010afc0 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.840+0000 7f6baca00700 1 -- 172.21.15.27:0/3621342789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba0011960 con 0x7f6bb010afc0 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.841+0000 7f6bb4c00700 1 -- 172.21.15.27:0/3621342789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 msgr2=0x7f6bb010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.841+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/3621342789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098a0 tx=0x7f6ba0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:05.901 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.841+0000 7f6bb4c00700 1 -- 172.21.15.27:0/3621342789 shutdown_connections 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.841+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/3621342789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb010b3e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.841+0000 7f6bb4c00700 1 -- 172.21.15.27:0/3621342789 >> 172.21.15.27:0/3621342789 conn(0x7f6bb007e3b0 msgr2=0x7f6bb007e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.842+0000 7f6bb4c00700 1 -- 172.21.15.27:0/3621342789 shutdown_connections 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.842+0000 7f6bb4c00700 1 -- 172.21.15.27:0/3621342789 wait complete. 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.843+0000 7f6bb4c00700 1 Processor -- start 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.843+0000 7f6bb4c00700 1 -- start start 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.843+0000 7f6bb4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.843+0000 7f6bb4c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb010f3f0 con 0x7f6bb010afc0 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.844+0000 7f6bade00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.844+0000 7f6bade00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33150/0 (socket says 172.21.15.27:33150) 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.844+0000 7f6bade00700 1 -- 172.21.15.27:0/2060013858 learned_addr learned my addr 172.21.15.27:0/2060013858 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.844+0000 7f6bade00700 1 -- 172.21.15.27:0/2060013858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb019f3d0 con 0x7f6bb010afc0 2024-04-26T21:00:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.844+0000 7f6bade00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0011670 tx=0x7f6ba0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.845+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba0003aa0 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.845+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb019f660 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.845+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba001b8d0 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.845+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ba00197f0 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.845+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb01a2350 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.846+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6ba001b440 con 0x7f6bb010afc0 2024-04-26T21:00:05.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.846+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb010f570 con 0x7f6bb010afc0 2024-04-26T21:00:05.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.847+0000 7f6ba6a00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 0x7f6b9403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:05.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.847+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ba004e010 con 0x7f6bb010afc0 2024-04-26T21:00:05.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.848+0000 7f6bad400700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 0x7f6b9403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.849+0000 7f6bad400700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 0x7f6b9403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006fd0 tx=0x7f6b9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:04.854+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ba001f030 con 0x7f6bb010afc0 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.095+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6bb0108bc0 con 0x7f6b940395b0 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.225+0000 7f6ba6a00700 1 -- 172.21.15.27:0/2060013858 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6bb0108bc0 con 0x7f6b940395b0 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 msgr2=0x7f6b9403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 0x7f6b9403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006fd0 tx=0x7f6b9c006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 msgr2=0x7f6bb019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0011670 tx=0x7f6ba0011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 shutdown_connections 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b940395b0 0x7f6b9403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 --2- 172.21.15.27:0/2060013858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bb010afc0 0x7f6bb019ee90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:05.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.228+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 >> 172.21.15.27:0/2060013858 conn(0x7f6bb007e3b0 msgr2=0x7f6bb01084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:05.907 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.229+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 shutdown_connections 2024-04-26T21:00:05.907 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:05.229+0000 7f6bb4c00700 1 -- 172.21.15.27:0/2060013858 wait complete. 2024-04-26T21:00:05.907 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-04-26T21:00:06.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:06 smithi027 ceph-mon[28085]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:06.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:06 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:06.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:06 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:07.749 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3pBilFHZ4iP/q/uJgg0fAe8txHSEZ8OA0S0z0Dtm977URC56wRBsO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714165207, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:00:07.749 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.503+0000 7fa1f4400700 1 Processor -- start 2024-04-26T21:00:07.749 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.504+0000 7fa1f4400700 1 -- start start 2024-04-26T21:00:07.749 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.504+0000 7fa1f4400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.504+0000 7fa1f4400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10b9b0 con 0x7fa1ec10afc0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.505+0000 7fa1f1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.505+0000 7fa1f1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33156/0 (socket says 172.21.15.27:33156) 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.505+0000 7fa1f1a00700 1 -- 172.21.15.27:0/1421818897 learned_addr learned my addr 172.21.15.27:0/1421818897 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.505+0000 7fa1f1a00700 1 -- 172.21.15.27:0/1421818897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec10c1c0 con 0x7fa1ec10afc0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.506+0000 7fa1f1a00700 1 --2- 172.21.15.27:0/1421818897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098a0 tx=0x7fa1e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b48f024d6306d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.506+0000 7fa1ebe00700 1 -- 172.21.15.27:0/1421818897 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0004160 con 0x7fa1ec10afc0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.506+0000 7fa1ebe00700 1 -- 172.21.15.27:0/1421818897 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1e0004300 con 0x7fa1ec10afc0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.506+0000 7fa1ebe00700 1 -- 172.21.15.27:0/1421818897 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0011960 con 0x7fa1ec10afc0 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.507+0000 7fa1f4400700 1 -- 172.21.15.27:0/1421818897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 msgr2=0x7fa1ec10b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.507+0000 7fa1f4400700 1 --2- 172.21.15.27:0/1421818897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098a0 tx=0x7fa1e0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:07.750 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.507+0000 7fa1f4400700 1 -- 172.21.15.27:0/1421818897 shutdown_connections 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.507+0000 7fa1f4400700 1 --2- 172.21.15.27:0/1421818897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec10b3e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.507+0000 7fa1f4400700 1 -- 172.21.15.27:0/1421818897 >> 172.21.15.27:0/1421818897 conn(0x7fa1ec07e3b0 msgr2=0x7fa1ec07e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 -- 172.21.15.27:0/1421818897 shutdown_connections 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 -- 172.21.15.27:0/1421818897 wait complete. 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 Processor -- start 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 -- start start 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.509+0000 7fa1f4400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10f3f0 con 0x7fa1ec10afc0 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.510+0000 7fa1f1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.510+0000 7fa1f1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33160/0 (socket says 172.21.15.27:33160) 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.510+0000 7fa1f1a00700 1 -- 172.21.15.27:0/2560224116 learned_addr learned my addr 172.21.15.27:0/2560224116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.510+0000 7fa1f1a00700 1 -- 172.21.15.27:0/2560224116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec19f3d0 con 0x7fa1ec10afc0 2024-04-26T21:00:07.751 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1f1a00700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0011670 tx=0x7fa1e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e001f070 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1e0003b20 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0019580 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec19f660 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.511+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a2350 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.512+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa1e0022070 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.513+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec10f570 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.513+0000 7fa1ea000700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 0x7fa1d403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.513+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1e004db70 con 0x7fa1ec10afc0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.513+0000 7fa1f1000700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 0x7fa1d403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.514+0000 7fa1f1000700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 0x7fa1d403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006fd0 tx=0x7fa1d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.519+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa1e001b440 con 0x7fa1ec10afc0 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:06.780+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa1ec064a80 con 0x7fa1d4039660 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.039+0000 7fa1ea000700 1 -- 172.21.15.27:0/2560224116 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa1ec064a80 con 0x7fa1d4039660 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 msgr2=0x7fa1d403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 0x7fa1d403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006fd0 tx=0x7fa1d8006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 msgr2=0x7fa1ec19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0011670 tx=0x7fa1e0011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 shutdown_connections 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1d4039660 0x7fa1d403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 --2- 172.21.15.27:0/2560224116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec10afc0 0x7fa1ec19ee90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 >> 172.21.15.27:0/2560224116 conn(0x7fa1ec07e3b0 msgr2=0x7fa1ec108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 shutdown_connections 2024-04-26T21:00:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:07.044+0000 7fa1f4400700 1 -- 172.21.15.27:0/2560224116 wait complete. 2024-04-26T21:00:07.754 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-04-26T21:00:08.369 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:08 smithi027 ceph-mon[28085]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:08.369 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:08.369 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:09.728 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:09 smithi027 ceph-mon[28085]: mgrmap e13: x(active, since 6s) 2024-04-26T21:00:09.729 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/466243617' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:00:10.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:00:10.199 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.367+0000 7fd6ed800700 1 Processor -- start 2024-04-26T21:00:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.368+0000 7fd6ed800700 1 -- start start 2024-04-26T21:00:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.368+0000 7fd6ed800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.368+0000 7fd6ed800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e807d870 con 0x7fd6e807ee40 2024-04-26T21:00:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.369+0000 7fd6e6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.369+0000 7fd6e6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33174/0 (socket says 172.21.15.27:33174) 2024-04-26T21:00:10.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.369+0000 7fd6e6a00700 1 -- 172.21.15.27:0/3632337675 learned_addr learned my addr 172.21.15.27:0/3632337675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:10.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.369+0000 7fd6e6a00700 1 -- 172.21.15.27:0/3632337675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e807d9f0 con 0x7fd6e807ee40 2024-04-26T21:00:10.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.370+0000 7fd6e6a00700 1 --2- 172.21.15.27:0/3632337675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80098a0 tx=0x7fd6d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbff023575db65b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:10.201 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.370+0000 7fd6e5600700 1 -- 172.21.15.27:0/3632337675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d8004160 con 0x7fd6e807ee40 2024-04-26T21:00:10.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.370+0000 7fd6e5600700 1 -- 172.21.15.27:0/3632337675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6d8004300 con 0x7fd6e807ee40 2024-04-26T21:00:10.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.370+0000 7fd6e5600700 1 -- 172.21.15.27:0/3632337675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d8011960 con 0x7fd6e807ee40 2024-04-26T21:00:10.202 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.371+0000 7fd6ed800700 1 -- 172.21.15.27:0/3632337675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 msgr2=0x7fd6e807d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:10.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.371+0000 7fd6ed800700 1 --2- 172.21.15.27:0/3632337675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80098a0 tx=0x7fd6d8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:10.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.371+0000 7fd6ed800700 1 -- 172.21.15.27:0/3632337675 shutdown_connections 2024-04-26T21:00:10.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.371+0000 7fd6ed800700 1 --2- 172.21.15.27:0/3632337675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e807d2a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:10.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.371+0000 7fd6ed800700 1 -- 172.21.15.27:0/3632337675 >> 172.21.15.27:0/3632337675 conn(0x7fd6e81043b0 msgr2=0x7fd6e81067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:10.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.372+0000 7fd6ed800700 1 -- 172.21.15.27:0/3632337675 shutdown_connections 2024-04-26T21:00:10.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.372+0000 7fd6ed800700 1 -- 172.21.15.27:0/3632337675 wait complete. 2024-04-26T21:00:10.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.373+0000 7fd6ed800700 1 Processor -- start 2024-04-26T21:00:10.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.373+0000 7fd6ed800700 1 -- start start 2024-04-26T21:00:10.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.373+0000 7fd6ed800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:10.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.373+0000 7fd6ed800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e810f330 con 0x7fd6e807ee40 2024-04-26T21:00:10.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.374+0000 7fd6e6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:10.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.374+0000 7fd6e6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33190/0 (socket says 172.21.15.27:33190) 2024-04-26T21:00:10.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.374+0000 7fd6e6a00700 1 -- 172.21.15.27:0/466243617 learned_addr learned my addr 172.21.15.27:0/466243617 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:10.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.374+0000 7fd6e6a00700 1 -- 172.21.15.27:0/466243617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e819f3e0 con 0x7fd6e807ee40 2024-04-26T21:00:10.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.374+0000 7fd6e6a00700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8011670 tx=0x7fd6d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:10.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.375+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d801f070 con 0x7fd6e807ee40 2024-04-26T21:00:10.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.375+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6d8003b20 con 0x7fd6e807ee40 2024-04-26T21:00:10.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.375+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d80195f0 con 0x7fd6e807ee40 2024-04-26T21:00:10.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.375+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e819f670 con 0x7fd6e807ee40 2024-04-26T21:00:10.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.375+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81a2360 con 0x7fd6e807ee40 2024-04-26T21:00:10.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.376+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd6d8022070 con 0x7fd6e807ee40 2024-04-26T21:00:10.208 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.376+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e8064a80 con 0x7fd6e807ee40 2024-04-26T21:00:10.209 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.377+0000 7fd6df400700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 0x7fd6cc03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.377+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6d804dd90 con 0x7fd6e807ee40 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.377+0000 7fd6e6000700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 0x7fd6cc03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.378+0000 7fd6e6000700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 0x7fd6cc03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006fd0 tx=0x7fd6d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.383+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6d801b490 con 0x7fd6e807ee40 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.620+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd6e81a2910 con 0x7fd6e807ee40 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.621+0000 7fd6df400700 1 -- 172.21.15.27:0/466243617 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd6d801baf0 con 0x7fd6e807ee40 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 msgr2=0x7fd6cc03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 0x7fd6cc03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006fd0 tx=0x7fd6d4006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 msgr2=0x7fd6e819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:10.210 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8011670 tx=0x7fd6d8011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:10.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 shutdown_connections 2024-04-26T21:00:10.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd6cc039700 0x7fd6cc03bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:10.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 --2- 172.21.15.27:0/466243617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6e807ee40 0x7fd6e819eea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:10.211 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.624+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 >> 172.21.15.27:0/466243617 conn(0x7fd6e81043b0 msgr2=0x7fd6e8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:10.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.625+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 shutdown_connections 2024-04-26T21:00:10.212 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:08.625+0000 7fd6ed800700 1 -- 172.21.15.27:0/466243617 wait complete. 2024-04-26T21:00:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:00:10.253 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-04-26T21:00:10.280 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:00:10.281 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-04-26T21:00:10.281 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-04-26T21:00:10.281 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:00:10.289 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-04-26T21:00:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-04-26T21:00:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-04-26T21:00:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: Password: dn6aj4za0x 2024-04-26T21:00:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:10.291 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config directory 2024-04-26T21:00:10.293 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.924+0000 7f2402600700 1 Processor -- start 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.924+0000 7f2402600700 1 -- start start 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.925+0000 7f2402600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.925+0000 7f2402600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10b9b0 con 0x7f23fc10afc0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.926+0000 7f23fb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.926+0000 7f23fb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46012/0 (socket says 172.21.15.27:46012) 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.926+0000 7f23fb400700 1 -- 172.21.15.27:0/2650507631 learned_addr learned my addr 172.21.15.27:0/2650507631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.926+0000 7f23fb400700 1 -- 172.21.15.27:0/2650507631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc10c1c0 con 0x7f23fc10afc0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.927+0000 7f23fb400700 1 --2- 172.21.15.27:0/2650507631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0098a0 tx=0x7f23ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7395b35d04d45c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.927+0000 7f23faa00700 1 -- 172.21.15.27:0/2650507631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ec004160 con 0x7f23fc10afc0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.927+0000 7f23faa00700 1 -- 172.21.15.27:0/2650507631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23ec004300 con 0x7f23fc10afc0 2024-04-26T21:00:11.734 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.927+0000 7f23faa00700 1 -- 172.21.15.27:0/2650507631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ec011960 con 0x7f23fc10afc0 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 -- 172.21.15.27:0/2650507631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 msgr2=0x7f23fc10b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 --2- 172.21.15.27:0/2650507631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0098a0 tx=0x7f23ec009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 -- 172.21.15.27:0/2650507631 shutdown_connections 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 --2- 172.21.15.27:0/2650507631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc10b3e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 -- 172.21.15.27:0/2650507631 >> 172.21.15.27:0/2650507631 conn(0x7f23fc07e3b0 msgr2=0x7f23fc07e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 -- 172.21.15.27:0/2650507631 shutdown_connections 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.928+0000 7f2402600700 1 -- 172.21.15.27:0/2650507631 wait complete. 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.929+0000 7f2402600700 1 Processor -- start 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.929+0000 7f2402600700 1 -- start start 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f2402600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f2402600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10f3f0 con 0x7f23fc10afc0 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f23fb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f23fb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46020/0 (socket says 172.21.15.27:46020) 2024-04-26T21:00:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f23fb400700 1 -- 172.21.15.27:0/2079875079 learned_addr learned my addr 172.21.15.27:0/2079875079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.930+0000 7f23fb400700 1 -- 172.21.15.27:0/2079875079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc19f3d0 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f23fb400700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23ec011670 tx=0x7f23ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ec01f070 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23ec003b20 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc19f660 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ec019580 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.931+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a2350 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.932+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc10f570 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.932+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f23ec022070 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.933+0000 7f23f8c00700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 0x7f23e003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.933+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23ec04dd30 con 0x7f23fc10afc0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.933+0000 7f23f3e00700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 0x7f23e003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.934+0000 7f23f3e00700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 0x7f23e003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006fd0 tx=0x7f23e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:11.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:10.939+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f23ec018350 con 0x7f23fc10afc0 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.167+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f23fc064a80 con 0x7f23fc10afc0 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.168+0000 7f23f8c00700 1 -- 172.21.15.27:0/2079875079 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f23ec01b440 con 0x7f23fc10afc0 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 msgr2=0x7f23e003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 0x7f23e003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006fd0 tx=0x7f23e4006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 msgr2=0x7f23fc19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23ec011670 tx=0x7f23ec011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 shutdown_connections 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f23e0039700 0x7f23e003bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 --2- 172.21.15.27:0/2079875079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23fc10afc0 0x7f23fc19ee90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 >> 172.21.15.27:0/2079875079 conn(0x7f23fc07e3b0 msgr2=0x7f23fc1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 shutdown_connections 2024-04-26T21:00:11.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:11.171+0000 7f2402600700 1 -- 172.21.15.27:0/2079875079 wait complete. 2024-04-26T21:00:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.368+0000 7f3d4d400700 1 Processor -- start 2024-04-26T21:00:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.369+0000 7f3d4d400700 1 -- start start 2024-04-26T21:00:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.369+0000 7f3d4d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.369+0000 7f3d4d400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112670 con 0x7f3d4810fcb0 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.369+0000 7f3d46800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.370+0000 7f3d46800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46034/0 (socket says 172.21.15.27:46034) 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.370+0000 7f3d46800700 1 -- 172.21.15.27:0/15492948 learned_addr learned my addr 172.21.15.27:0/15492948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.370+0000 7f3d46800700 1 -- 172.21.15.27:0/15492948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481127f0 con 0x7f3d4810fcb0 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.370+0000 7f3d46800700 1 --2- 172.21.15.27:0/15492948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d380098a0 tx=0x7f3d38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f56b83f0f8b6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.371+0000 7f3d45400700 1 -- 172.21.15.27:0/15492948 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d38004160 con 0x7f3d4810fcb0 2024-04-26T21:00:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.371+0000 7f3d45400700 1 -- 172.21.15.27:0/15492948 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d38004300 con 0x7f3d4810fcb0 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.371+0000 7f3d45400700 1 -- 172.21.15.27:0/15492948 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d38011960 con 0x7f3d4810fcb0 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.371+0000 7f3d4d400700 1 -- 172.21.15.27:0/15492948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 msgr2=0x7f3d481120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.371+0000 7f3d4d400700 1 --2- 172.21.15.27:0/15492948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d380098a0 tx=0x7f3d38009bb0 comp rx=0 tx=0).stop 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.372+0000 7f3d4d400700 1 -- 172.21.15.27:0/15492948 shutdown_connections 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.372+0000 7f3d4d400700 1 --2- 172.21.15.27:0/15492948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481120a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.372+0000 7f3d4d400700 1 -- 172.21.15.27:0/15492948 >> 172.21.15.27:0/15492948 conn(0x7f3d4807e3d0 msgr2=0x7f3d480807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.372+0000 7f3d4d400700 1 -- 172.21.15.27:0/15492948 shutdown_connections 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.372+0000 7f3d4d400700 1 -- 172.21.15.27:0/15492948 wait complete. 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.373+0000 7f3d4d400700 1 Processor -- start 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.373+0000 7f3d4d400700 1 -- start start 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.373+0000 7f3d4d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.373+0000 7f3d4d400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48113850 con 0x7f3d4810fcb0 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.374+0000 7f3d46800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.374+0000 7f3d46800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46038/0 (socket says 172.21.15.27:46038) 2024-04-26T21:00:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.374+0000 7f3d46800700 1 -- 172.21.15.27:0/402146994 learned_addr learned my addr 172.21.15.27:0/402146994 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.374+0000 7f3d46800700 1 -- 172.21.15.27:0/402146994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a38b0 con 0x7f3d4810fcb0 2024-04-26T21:00:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.374+0000 7f3d46800700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d38011670 tx=0x7f3d38011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.375+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3801f070 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.375+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d38003b20 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.375+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d380195f0 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.375+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a3b40 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.375+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a6830 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.376+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3d38022070 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.376+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d48064a80 con 0x7f3d4810fcb0 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.377+0000 7f3d3f400700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 0x7f3d2c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:13.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.377+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d3804dd60 con 0x7f3d4810fcb0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.377+0000 7f3d45e00700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 0x7f3d2c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.378+0000 7f3d45e00700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 0x7f3d2c03bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.382+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d3801b490 con 0x7f3d4810fcb0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.732+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3d481a6a00 con 0x7f3d4810fcb0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.738+0000 7f3d3f400700 1 -- 172.21.15.27:0/402146994 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3d3801baf0 con 0x7f3d4810fcb0 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 msgr2=0x7f3d2c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 0x7f3d2c03bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 msgr2=0x7f3d481a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d38011670 tx=0x7f3d38011a30 comp rx=0 tx=0).stop 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 shutdown_connections 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3d2c0396a0 0x7f3d2c03bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 --2- 172.21.15.27:0/402146994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d4810fcb0 0x7f3d481a3370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 >> 172.21.15.27:0/402146994 conn(0x7f3d4807e3d0 msgr2=0x7f3d4807eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 shutdown_connections 2024-04-26T21:00:13.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-04-26T21:00:12.741+0000 7f3d4d400700 1 -- 172.21.15.27:0/402146994 wait complete. 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:00:13.466 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:13.466 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-04-26T21:00:13.565 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:00:13.565 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:00:13.566 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:00:13.616 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:00:13.616 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:00:13.616 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:00:13.659 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:00:13.660 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:00:13.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:00:13.787 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:00:13.788 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:00:13.788 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:00:13.836 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:00:13.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:00:13.988 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:13 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/402146994' entity='client.admin' 2024-04-26T21:00:13.995 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:00:14.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:00:14.182 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:00:14.241 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:00:14.385 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzD8Z83K1dCIvfwgBM+w2Mg2G+ZouIxYiK7Q2TyPhY2PUN6yV7MgE33GKOv+VPs45X618TMecGGdn/9jTZRWxbFqObUoGAEx7iXWH8EA6lkI2gF/XzcImheDPw90rq8BwaKoGdyLRUAAtjp0njffPOehFfeEbugrZ8Po/zblK882bW363hPw85fCmP8oHcTrfH++NHOBXAgZZ77K853GD11S+nYbWktoqO57yn6bLCkYd1W+u2+WtotNqke53RJ9qUda6klYakL3PE+A3rzZ606J7dyn1aX11eY6AWwJmgt0nyyT9PEMIY/v/y/wPRFDztxNc8uFEr2htv5xkJq+I73NUTpNvVs6An4s/a+VNjtOe/r92/Z4BdiDnRiSL2v6s8z5A3BLUoZidaD88dEOeFUmiXfwD/IlgWa4F6P4m6DVFQuGNNJrW8BkmIkRWqkkabJspfx0MQo32WaavW/NfxUQ+i7VU3VrXdmxFLq3g9pf+k7oncYxbj4UnhS6iSzC0= ceph-a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:00:14.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:00:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 -- 172.21.15.27:0/3674772102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 msgr2=0x7f4a441030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 --2- 172.21.15.27:0/3674772102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a441030f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009910 tx=0x7f4a34009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 -- 172.21.15.27:0/3674772102 shutdown_connections 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 --2- 172.21.15.27:0/3674772102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a441030f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 -- 172.21.15.27:0/3674772102 >> 172.21.15.27:0/3674772102 conn(0x7f4a440fe2d0 msgr2=0x7f4a441006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:17.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 -- 172.21.15.27:0/3674772102 shutdown_connections 2024-04-26T21:00:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.517+0000 7f4a43e00700 1 -- 172.21.15.27:0/3674772102 wait complete. 2024-04-26T21:00:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.519+0000 7f4a43e00700 1 Processor -- start 2024-04-26T21:00:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.519+0000 7f4a43e00700 1 -- start start 2024-04-26T21:00:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.520+0000 7f4a43e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.520+0000 7f4a43e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44108ac0 con 0x7f4a44102cd0 2024-04-26T21:00:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.520+0000 7f4a42a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.520+0000 7f4a42a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46044/0 (socket says 172.21.15.27:46044) 2024-04-26T21:00:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.520+0000 7f4a42a00700 1 -- 172.21.15.27:0/220349482 learned_addr learned my addr 172.21.15.27:0/220349482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.521+0000 7f4a42a00700 1 -- 172.21.15.27:0/220349482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4419a980 con 0x7f4a44102cd0 2024-04-26T21:00:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.521+0000 7f4a42a00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a34005120 tx=0x7f4a34005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.521+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a34027070 con 0x7f4a44102cd0 2024-04-26T21:00:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.521+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a3400f460 con 0x7f4a44102cd0 2024-04-26T21:00:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.521+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a34017440 con 0x7f4a44102cd0 2024-04-26T21:00:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.522+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4419ac10 con 0x7f4a44102cd0 2024-04-26T21:00:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.522+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4419d900 con 0x7f4a44102cd0 2024-04-26T21:00:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.523+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a44109240 con 0x7f4a44102cd0 2024-04-26T21:00:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.523+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4a3402a070 con 0x7f4a44102cd0 2024-04-26T21:00:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.524+0000 7f4a48a00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 0x7f4a2803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.524+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4a3404d9f0 con 0x7f4a44102cd0 2024-04-26T21:00:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.524+0000 7f4a42000700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 0x7f4a2803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.525+0000 7f4a42000700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 0x7f4a2803bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.530+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4a34013070 con 0x7f4a44102cd0 2024-04-26T21:00:17.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.770+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4a4402e5a0 con 0x7f4a44102cd0 2024-04-26T21:00:17.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.774+0000 7f4a48a00700 1 -- 172.21.15.27:0/220349482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4a340292c0 con 0x7f4a44102cd0 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.777+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 msgr2=0x7f4a2803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.777+0000 7f4a43e00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 0x7f4a2803bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.777+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 msgr2=0x7f4a4419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.777+0000 7f4a43e00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a34005120 tx=0x7f4a34005810 comp rx=0 tx=0).stop 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 shutdown_connections 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4a28039700 0x7f4a2803bbc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 --2- 172.21.15.27:0/220349482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a44102cd0 0x7f4a4419a440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 >> 172.21.15.27:0/220349482 conn(0x7f4a440fe2d0 msgr2=0x7f4a441006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 shutdown_connections 2024-04-26T21:00:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:17.778+0000 7f4a43e00700 1 -- 172.21.15.27:0/220349482 wait complete. 2024-04-26T21:00:17.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:17.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:17.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:00:17.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:17.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:17.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:17.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/220349482' entity='client.admin' 2024-04-26T21:00:17.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:17.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:18.633 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:00:18.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:00:19.090 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 -- 172.21.15.27:0/1702474391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 msgr2=0x7fb71c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 --2- 172.21.15.27:0/1702474391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c103280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009910 tx=0x7fb70c009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 -- 172.21.15.27:0/1702474391 shutdown_connections 2024-04-26T21:00:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 --2- 172.21.15.27:0/1702474391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c103280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 -- 172.21.15.27:0/1702474391 >> 172.21.15.27:0/1702474391 conn(0x7fb71c0fe470 msgr2=0x7fb71c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 -- 172.21.15.27:0/1702474391 shutdown_connections 2024-04-26T21:00:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.831+0000 7fb721c00700 1 -- 172.21.15.27:0/1702474391 wait complete. 2024-04-26T21:00:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.832+0000 7fb721c00700 1 Processor -- start 2024-04-26T21:00:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.832+0000 7fb721c00700 1 -- start start 2024-04-26T21:00:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.832+0000 7fb721c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.832+0000 7fb721c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c108c10 con 0x7fb71c102e60 2024-04-26T21:00:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.833+0000 7fb71aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.833+0000 7fb71aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45842/0 (socket says 172.21.15.27:45842) 2024-04-26T21:00:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.833+0000 7fb71aa00700 1 -- 172.21.15.27:0/3019937844 learned_addr learned my addr 172.21.15.27:0/3019937844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.833+0000 7fb71aa00700 1 -- 172.21.15.27:0/3019937844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c198950 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb71aa00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005120 tx=0x7fb70c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c027070 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb70c00f460 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c017440 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c198be0 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.834+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c19b8d0 con 0x7fb71c102e60 2024-04-26T21:00:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.835+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c109390 con 0x7fb71c102e60 2024-04-26T21:00:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.835+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb70c02a070 con 0x7fb71c102e60 2024-04-26T21:00:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.836+0000 7fb713e00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 0x7fb70003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.836+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb70c04d950 con 0x7fb71c102e60 2024-04-26T21:00:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.836+0000 7fb712000700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 0x7fb70003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.837+0000 7fb712000700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 0x7fb70003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:21.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:21.842+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb70c013070 con 0x7fb71c102e60 2024-04-26T21:00:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.079+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb71c02e5a0 con 0x7fb7000396b0 2024-04-26T21:00:22.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.087+0000 7fb713e00700 1 -- 172.21.15.27:0/3019937844 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb71c02e5a0 con 0x7fb7000396b0 2024-04-26T21:00:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 msgr2=0x7fb70003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 0x7fb70003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 msgr2=0x7fb71c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005120 tx=0x7fb70c005810 comp rx=0 tx=0).stop 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 shutdown_connections 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb7000396b0 0x7fb70003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 --2- 172.21.15.27:0/3019937844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c102e60 0x7fb71c198410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.090+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 >> 172.21.15.27:0/3019937844 conn(0x7fb71c0fe470 msgr2=0x7fb71c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.091+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 shutdown_connections 2024-04-26T21:00:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:22.091+0000 7fb721c00700 1 -- 172.21.15.27:0/3019937844 wait complete. 2024-04-26T21:00:22.878 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi094 2024-04-26T21:00:22.878 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T21:00:22.878 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:00:22.936 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T21:00:22.937 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:23.034 INFO:tasks.cephadm:Adding host smithi094 to orchestrator... 2024-04-26T21:00:23.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch host add smithi094 2024-04-26T21:00:23.474 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:23.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:23 smithi027 ceph-mon[28085]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:00:24.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:24 smithi027 ceph-mon[28085]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:00:24.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:24 smithi027 ceph-mon[28085]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:25.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:25 smithi027 ceph-mon[28085]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:25.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:25 smithi027 ceph-mon[28085]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:00:25.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:25.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:25.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 -- 172.21.15.27:0/603029707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 msgr2=0x7ff6e0104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/603029707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e0104470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009910 tx=0x7ff6d0009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 -- 172.21.15.27:0/603029707 shutdown_connections 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/603029707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e0104470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 -- 172.21.15.27:0/603029707 >> 172.21.15.27:0/603029707 conn(0x7ff6e00fd820 msgr2=0x7ff6e00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 -- 172.21.15.27:0/603029707 shutdown_connections 2024-04-26T21:00:25.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.681+0000 7ff6e5e00700 1 -- 172.21.15.27:0/603029707 wait complete. 2024-04-26T21:00:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.682+0000 7ff6e5e00700 1 Processor -- start 2024-04-26T21:00:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6e5e00700 1 -- start start 2024-04-26T21:00:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6e5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:25.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6e5e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e010d240 con 0x7ff6e0104090 2024-04-26T21:00:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6dea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6dea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45864/0 (socket says 172.21.15.27:45864) 2024-04-26T21:00:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.683+0000 7ff6dea00700 1 -- 172.21.15.27:0/2781648806 learned_addr learned my addr 172.21.15.27:0/2781648806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:25.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.684+0000 7ff6dea00700 1 -- 172.21.15.27:0/2781648806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e01a0c10 con 0x7ff6e0104090 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.684+0000 7ff6dea00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005060 tx=0x7ff6d0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.684+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d0026070 con 0x7ff6e0104090 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.684+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6d000b750 con 0x7ff6e0104090 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.684+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e01a0ea0 con 0x7ff6e0104090 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.685+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d0003850 con 0x7ff6e0104090 2024-04-26T21:00:25.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.685+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e019a890 con 0x7ff6e0104090 2024-04-26T21:00:25.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.686+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff6d0029070 con 0x7ff6e0104090 2024-04-26T21:00:25.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.686+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e006bb50 con 0x7ff6e0104090 2024-04-26T21:00:25.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.686+0000 7ff6e4a00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 0x7ff6c403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:25.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.686+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6d001e070 con 0x7ff6e0104090 2024-04-26T21:00:25.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.687+0000 7ff6d6000700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 0x7ff6c403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:25.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.687+0000 7ff6d6000700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 0x7ff6c403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:25.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.692+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6d0013070 con 0x7ff6e0104090 2024-04-26T21:00:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:25.926+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}) -- 0x7ff6e019bdc0 con 0x7ff6c40396b0 2024-04-26T21:00:27.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:27 smithi027 ceph-mon[28085]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:27.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:27 smithi027 ceph-mon[28085]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:28.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:28 smithi027 ceph-mon[28085]: Deploying cephadm binary to smithi094 2024-04-26T21:00:29.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:29 smithi027 ceph-mon[28085]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:31.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:31 smithi027 ceph-mon[28085]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:32.321 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi094' with addr '172.21.15.94' 2024-04-26T21:00:32.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.318+0000 7ff6e4a00700 1 -- 172.21.15.27:0/2781648806 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff6e019bdc0 con 0x7ff6c40396b0 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.322+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 msgr2=0x7ff6c403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.322+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 0x7ff6c403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.322+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 msgr2=0x7ff6e01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.322+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005060 tx=0x7ff6d0005710 comp rx=0 tx=0).stop 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 shutdown_connections 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff6c40396b0 0x7ff6c403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 --2- 172.21.15.27:0/2781648806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6e0104090 0x7ff6e01a06d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:32.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 >> 172.21.15.27:0/2781648806 conn(0x7ff6e00fd820 msgr2=0x7ff6e0109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:32.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 shutdown_connections 2024-04-26T21:00:32.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:32.323+0000 7ff6e5e00700 1 -- 172.21.15.27:0/2781648806 wait complete. 2024-04-26T21:00:33.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:00:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:33.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:33 smithi027 ceph-mon[28085]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:33.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:33.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:33 smithi027 ceph-mon[28085]: Added host smithi094 2024-04-26T21:00:33.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:33.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:35.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:35 smithi027 ceph-mon[28085]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:35.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.588+0000 7f15afa00700 1 -- 172.21.15.27:0/2749237789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 msgr2=0x7f15a81092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.588+0000 7f15afa00700 1 --2- 172.21.15.27:0/2749237789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a81092a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f159c009910 tx=0x7f159c009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.588+0000 7f15afa00700 1 -- 172.21.15.27:0/2749237789 shutdown_connections 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.588+0000 7f15afa00700 1 --2- 172.21.15.27:0/2749237789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a81092a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.588+0000 7f15afa00700 1 -- 172.21.15.27:0/2749237789 >> 172.21.15.27:0/2749237789 conn(0x7f15a8077880 msgr2=0x7f15a8077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.589+0000 7f15afa00700 1 -- 172.21.15.27:0/2749237789 shutdown_connections 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.589+0000 7f15afa00700 1 -- 172.21.15.27:0/2749237789 wait complete. 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.590+0000 7f15afa00700 1 Processor -- start 2024-04-26T21:00:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.590+0000 7f15afa00700 1 -- start start 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.590+0000 7f15afa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.590+0000 7f15afa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810d1b0 con 0x7f15a8108ec0 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.590+0000 7f15ad000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.591+0000 7f15ad000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49824/0 (socket says 172.21.15.27:49824) 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.591+0000 7f15ad000700 1 -- 172.21.15.27:0/30120497 learned_addr learned my addr 172.21.15.27:0/30120497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:35.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.591+0000 7f15ad000700 1 -- 172.21.15.27:0/30120497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a819e940 con 0x7f15a8108ec0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.591+0000 7f15ad000700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f159c006010 tx=0x7f159c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.591+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f159c026070 con 0x7f15a8108ec0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.592+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f159c00b6f0 con 0x7f15a8108ec0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.592+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f159c003810 con 0x7f15a8108ec0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.592+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a819ec30 con 0x7f15a8108ec0 2024-04-26T21:00:35.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.592+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81985c0 con 0x7f15a8108ec0 2024-04-26T21:00:35.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.593+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f159c029070 con 0x7f15a8108ec0 2024-04-26T21:00:35.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.593+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a8104570 con 0x7f15a8108ec0 2024-04-26T21:00:35.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.594+0000 7f15a5600700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 0x7f159003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:35.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.594+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f159c01e070 con 0x7f15a8108ec0 2024-04-26T21:00:35.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.594+0000 7f15a7e00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 0x7f159003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:35.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.595+0000 7f15a7e00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 0x7f159003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1594006fd0 tx=0x7f1594006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:35.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.601+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f159c013070 con 0x7f15a8108ec0 2024-04-26T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.844+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15a8199af0 con 0x7f15900396b0 2024-04-26T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.846+0000 7f15a5600700 1 -- 172.21.15.27:0/30120497 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f15a8199af0 con 0x7f15900396b0 2024-04-26T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}] 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.849+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 msgr2=0x7f159003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.849+0000 7f15afa00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 0x7f159003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1594006fd0 tx=0x7f1594006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.849+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 msgr2=0x7f15a819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.849+0000 7f15afa00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f159c006010 tx=0x7f159c005710 comp rx=0 tx=0).stop 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.849+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 shutdown_connections 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.850+0000 7f15afa00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f15900396b0 0x7f159003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.850+0000 7f15afa00700 1 --2- 172.21.15.27:0/30120497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15a8108ec0 0x7f15a819e400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.850+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 >> 172.21.15.27:0/30120497 conn(0x7f15a8077880 msgr2=0x7f15a80ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.850+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 shutdown_connections 2024-04-26T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:35.850+0000 7f15afa00700 1 -- 172.21.15.27:0/30120497 wait complete. 2024-04-26T21:00:36.640 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-04-26T21:00:36.641 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:00:36.641 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:00:36.699 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:00:36.699 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:36.796 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-04-26T21:00:36.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch host add smithi204 2024-04-26T21:00:37.258 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:37.530 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:37 smithi027 ceph-mon[28085]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:00:37.530 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:37 smithi027 ceph-mon[28085]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:37.530 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:39.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 -- 172.21.15.27:0/1749271211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 msgr2=0x7f917c10e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 --2- 172.21.15.27:0/1749271211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c10e7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f916c009910 tx=0x7f916c009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 -- 172.21.15.27:0/1749271211 shutdown_connections 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 --2- 172.21.15.27:0/1749271211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c10e7c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 -- 172.21.15.27:0/1749271211 >> 172.21.15.27:0/1749271211 conn(0x7f917c078cb0 msgr2=0x7f917c07b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 -- 172.21.15.27:0/1749271211 shutdown_connections 2024-04-26T21:00:39.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.236+0000 7f9181c00700 1 -- 172.21.15.27:0/1749271211 wait complete. 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.237+0000 7f9181c00700 1 Processor -- start 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.237+0000 7f9181c00700 1 -- start start 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f9181c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f9181c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f917c113800 con 0x7f917c10e3e0 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f917aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f917aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36062/0 (socket says 172.21.15.27:36062) 2024-04-26T21:00:39.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f917aa00700 1 -- 172.21.15.27:0/1561939996 learned_addr learned my addr 172.21.15.27:0/1561939996 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:39.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.238+0000 7f917aa00700 1 -- 172.21.15.27:0/1561939996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c19f2d0 con 0x7f917c10e3e0 2024-04-26T21:00:39.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f917aa00700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c005060 tx=0x7f916c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:39.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f916c026070 con 0x7f917c10e3e0 2024-04-26T21:00:39.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f916c00b6f0 con 0x7f917c10e3e0 2024-04-26T21:00:39.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f916c003810 con 0x7f917c10e3e0 2024-04-26T21:00:39.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917c19f5c0 con 0x7f917c10e3e0 2024-04-26T21:00:39.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.239+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917c1a3660 con 0x7f917c10e3e0 2024-04-26T21:00:39.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.240+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917c10ab70 con 0x7f917c10e3e0 2024-04-26T21:00:39.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.240+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f916c029070 con 0x7f917c10e3e0 2024-04-26T21:00:39.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.241+0000 7f9173400700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 0x7f916003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:39.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.241+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f916c01e070 con 0x7f917c10e3e0 2024-04-26T21:00:39.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.241+0000 7f917a000700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 0x7f916003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:39.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.242+0000 7f917a000700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 0x7f916003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9168006fd0 tx=0x7f9168006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:39.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.247+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f916c013070 con 0x7f917c10e3e0 2024-04-26T21:00:39.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:39 smithi027 ceph-mon[28085]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:39.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:39.483+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7f917c1a00f0 con 0x7f91600396b0 2024-04-26T21:00:40.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:40 smithi027 ceph-mon[28085]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:42.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:42 smithi027 ceph-mon[28085]: Deploying cephadm binary to smithi204 2024-04-26T21:00:42.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:42 smithi027 ceph-mon[28085]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:00:43.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:43 smithi027 ceph-mon[28085]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:45.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:45 smithi027 ceph-mon[28085]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:45.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:45 smithi027 ceph-mon[28085]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:00:45.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.756+0000 7f9173400700 1 -- 172.21.15.27:0/1561939996 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f917c1a00f0 con 0x7f91600396b0 2024-04-26T21:00:45.758 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-04-26T21:00:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 msgr2=0x7f916003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 0x7f916003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9168006fd0 tx=0x7f9168006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 msgr2=0x7f917c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c005060 tx=0x7f916c005710 comp rx=0 tx=0).stop 2024-04-26T21:00:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 shutdown_connections 2024-04-26T21:00:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f91600396b0 0x7f916003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 --2- 172.21.15.27:0/1561939996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f917c10e3e0 0x7f917c19ed90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.760+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 >> 172.21.15.27:0/1561939996 conn(0x7f917c078cb0 msgr2=0x7f917c079a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.761+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 shutdown_connections 2024-04-26T21:00:45.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:45.761+0000 7f9181c00700 1 -- 172.21.15.27:0/1561939996 wait complete. 2024-04-26T21:00:46.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:46 smithi027 ceph-mon[28085]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:46.442 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:46.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:00:47.011 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: Added host smithi204 2024-04-26T21:00:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:00:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 -- 172.21.15.27:0/1902640559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 msgr2=0x7f3b8c105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 --2- 172.21.15.27:0/1902640559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c105680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009910 tx=0x7f3b7c009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 -- 172.21.15.27:0/1902640559 shutdown_connections 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 --2- 172.21.15.27:0/1902640559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c105680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 -- 172.21.15.27:0/1902640559 >> 172.21.15.27:0/1902640559 conn(0x7f3b8c100b50 msgr2=0x7f3b8c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 -- 172.21.15.27:0/1902640559 shutdown_connections 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.188+0000 7f3b91200700 1 -- 172.21.15.27:0/1902640559 wait complete. 2024-04-26T21:00:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.190+0000 7f3b91200700 1 Processor -- start 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.190+0000 7f3b91200700 1 -- start start 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.190+0000 7f3b91200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.190+0000 7f3b91200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10f3d0 con 0x7f3b8c1052a0 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.191+0000 7f3b89e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.191+0000 7f3b89e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55438/0 (socket says 172.21.15.27:55438) 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.191+0000 7f3b89e00700 1 -- 172.21.15.27:0/3039262319 learned_addr learned my addr 172.21.15.27:0/3039262319 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.191+0000 7f3b89e00700 1 -- 172.21.15.27:0/3039262319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c1a0bc0 con 0x7f3b8c1052a0 2024-04-26T21:00:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.191+0000 7f3b89e00700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006010 tx=0x7f3b7c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.192+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b7c026070 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.192+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1a0eb0 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.192+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b7c00b6f0 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.192+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b7c003810 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.192+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c19a800 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.193+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c106780 con 0x7f3b8c1052a0 2024-04-26T21:00:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.193+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3b7c029070 con 0x7f3b8c1052a0 2024-04-26T21:00:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.194+0000 7f3b82a00700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 0x7f3b7003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.194+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b7c01e070 con 0x7f3b8c1052a0 2024-04-26T21:00:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.194+0000 7f3b89400700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 0x7f3b7003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:49.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.195+0000 7f3b89400700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 0x7f3b7003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b78006fd0 tx=0x7f3b78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:49.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.200+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3b7c013070 con 0x7f3b8c1052a0 2024-04-26T21:00:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.438+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b8c19bd30 con 0x7f3b70039700 2024-04-26T21:00:49.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:49 smithi027 ceph-mon[28085]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:49.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:49.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.439+0000 7f3b82a00700 1 -- 172.21.15.27:0/3039262319 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3b8c19bd30 con 0x7f3b70039700 2024-04-26T21:00:49.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:00:49.441 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-04-26T21:00:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 msgr2=0x7f3b7003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 0x7f3b7003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b78006fd0 tx=0x7f3b78006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 msgr2=0x7f3b8c1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006010 tx=0x7f3b7c005710 comp rx=0 tx=0).stop 2024-04-26T21:00:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 shutdown_connections 2024-04-26T21:00:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3b70039700 0x7f3b7003bbc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 --2- 172.21.15.27:0/3039262319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b8c1052a0 0x7f3b8c1a0680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 >> 172.21.15.27:0/3039262319 conn(0x7f3b8c100b50 msgr2=0x7f3b8c106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 shutdown_connections 2024-04-26T21:00:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:49.444+0000 7f3b91200700 1 -- 172.21.15.27:0/3039262319 wait complete. 2024-04-26T21:00:50.246 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:00:50.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:00:50.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:50 smithi027 ceph-mon[28085]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:00:50.691 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:00:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:51 smithi027 ceph-mon[28085]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:52.430 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:52 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 -- 172.21.15.27:0/4135608695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 msgr2=0x7f99a8109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 --2- 172.21.15.27:0/4135608695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a8109100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f999c009910 tx=0x7f999c009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 -- 172.21.15.27:0/4135608695 shutdown_connections 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 --2- 172.21.15.27:0/4135608695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a8109100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 -- 172.21.15.27:0/4135608695 >> 172.21.15.27:0/4135608695 conn(0x7f99a80776d0 msgr2=0x7f99a8077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 -- 172.21.15.27:0/4135608695 shutdown_connections 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.605+0000 7f99ae800700 1 -- 172.21.15.27:0/4135608695 wait complete. 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.606+0000 7f99ae800700 1 Processor -- start 2024-04-26T21:00:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ae800700 1 -- start start 2024-04-26T21:00:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ae800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ae800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810d000 con 0x7f99a8108d20 2024-04-26T21:00:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ad400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ad400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55452/0 (socket says 172.21.15.27:55452) 2024-04-26T21:00:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.607+0000 7f99ad400700 1 -- 172.21.15.27:0/2558168279 learned_addr learned my addr 172.21.15.27:0/2558168279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:00:52.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99ad400700 1 -- 172.21.15.27:0/2558168279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a0aa0 con 0x7f99a8108d20 2024-04-26T21:00:52.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99ad400700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f999c026070 con 0x7f99a8108d20 2024-04-26T21:00:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f999c00b6f0 con 0x7f99a8108d20 2024-04-26T21:00:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f999c003810 con 0x7f99a8108d20 2024-04-26T21:00:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.608+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a0d90 con 0x7f99a8108d20 2024-04-26T21:00:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.609+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a819a800 con 0x7f99a8108d20 2024-04-26T21:00:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.610+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a81043d0 con 0x7f99a8108d20 2024-04-26T21:00:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.610+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f999c029070 con 0x7f99a8108d20 2024-04-26T21:00:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.610+0000 7f99a6000700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 0x7f999003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.611+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f999c01e070 con 0x7f99a8108d20 2024-04-26T21:00:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.611+0000 7f99aca00700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 0x7f999003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:52.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.612+0000 7f99aca00700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 0x7f999003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9994006fd0 tx=0x7f9994006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:52.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.616+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f999c013070 con 0x7f99a8108d20 2024-04-26T21:00:52.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:52.851+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f99a819bb60 con 0x7f99a8108d20 2024-04-26T21:00:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.135+0000 7f99a6000700 1 -- 172.21.15.27:0/2558168279 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f999c02ade0 con 0x7f99a8108d20 2024-04-26T21:00:53.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 msgr2=0x7f999003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:53.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 0x7f999003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9994006fd0 tx=0x7f9994006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:53.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 msgr2=0x7f99a81a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:53.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c005710 comp rx=0 tx=0).stop 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 shutdown_connections 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99900396b0 0x7f999003bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 --2- 172.21.15.27:0/2558168279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a8108d20 0x7f99a81a0560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 >> 172.21.15.27:0/2558168279 conn(0x7f99a80776d0 msgr2=0x7f99a8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.139+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 shutdown_connections 2024-04-26T21:00:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:00:53.140+0000 7f99ae800700 1 -- 172.21.15.27:0/2558168279 wait complete. 2024-04-26T21:00:53.142 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-04-26T21:00:53.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:53 smithi027 ceph-mon[28085]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:53.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:53 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:00:53.962 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-04-26T21:00:53.963 INFO:tasks.cephadm:Adding mon.b on smithi094 2024-04-26T21:00:53.963 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-04-26T21:00:53.963 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c' 2024-04-26T21:00:54.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:54 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:00:54.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:54 smithi027 ceph-mon[28085]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:00:54.373 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:00:54.480 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:00:55.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:55 smithi027 ceph-mon[28085]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:57.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 -- 172.21.15.204:0/3970642363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 msgr2=0x7fedd8101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:57.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 --2- 172.21.15.204:0/3970642363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd8101b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009910 tx=0x7fedcc009c20 comp rx=0 tx=0).stop 2024-04-26T21:00:57.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 -- 172.21.15.204:0/3970642363 shutdown_connections 2024-04-26T21:00:57.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 --2- 172.21.15.204:0/3970642363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd8101b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:57.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 -- 172.21.15.204:0/3970642363 >> 172.21.15.204:0/3970642363 conn(0x7fedd80fd320 msgr2=0x7fedd80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:57.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.187+0000 7feddfe00700 1 -- 172.21.15.204:0/3970642363 shutdown_connections 2024-04-26T21:00:57.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.188+0000 7feddfe00700 1 -- 172.21.15.204:0/3970642363 wait complete. 2024-04-26T21:00:57.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.189+0000 7feddfe00700 1 Processor -- start 2024-04-26T21:00:57.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.189+0000 7feddfe00700 1 -- start start 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.189+0000 7feddfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.189+0000 7feddfe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedd810ae60 con 0x7fedd8101760 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.190+0000 7feddd400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.190+0000 7feddd400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.204:59970/0 (socket says 172.21.15.204:59970) 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.190+0000 7feddd400700 1 -- 172.21.15.204:0/3212053656 learned_addr learned my addr 172.21.15.204:0/3212053656 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:00:57.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.190+0000 7feddd400700 1 -- 172.21.15.204:0/3212053656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd819aae0 con 0x7fedd8101760 2024-04-26T21:00:57.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.191+0000 7feddd400700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedcc000c00 tx=0x7fedcc005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:57.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.191+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedcc027070 con 0x7fedd8101760 2024-04-26T21:00:57.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.191+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedcc00f460 con 0x7fedd8101760 2024-04-26T21:00:57.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.191+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedcc017440 con 0x7fedd8101760 2024-04-26T21:00:57.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.191+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd819ad70 con 0x7fedd8101760 2024-04-26T21:00:57.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.192+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd819da60 con 0x7fedd8101760 2024-04-26T21:00:57.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.193+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd810b5e0 con 0x7fedd8101760 2024-04-26T21:00:57.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.193+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fedcc02a070 con 0x7fedd8101760 2024-04-26T21:00:57.195 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.194+0000 7fedd6000700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 0x7fedc003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:00:57.195 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.194+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fedcc04d9e0 con 0x7fedd8101760 2024-04-26T21:00:57.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.195+0000 7feddca00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 0x7fedc003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:00:57.197 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.196+0000 7feddca00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 0x7fedc003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006fd0 tx=0x7fedc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:00:57.213 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.212+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fedcc013070 con 0x7fedd8101760 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:57.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:57.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:56 smithi027 ceph-mon[28085]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:00:57.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.454+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7fedd802e720 con 0x7fedc0039650 2024-04-26T21:00:57.470 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.469+0000 7fedd6000700 1 -- 172.21.15.204:0/3212053656 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fedd802e720 con 0x7fedc0039650 2024-04-26T21:00:57.471 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-04-26T21:00:57.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 msgr2=0x7fedc003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:57.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 0x7fedc003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006fd0 tx=0x7fedc4006e40 comp rx=0 tx=0).stop 2024-04-26T21:00:57.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 msgr2=0x7fedd819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedcc000c00 tx=0x7fedcc005810 comp rx=0 tx=0).stop 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 shutdown_connections 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fedc0039650 0x7fedc003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 --2- 172.21.15.204:0/3212053656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedd8101760 0x7fedd819a5a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 >> 172.21.15.204:0/3212053656 conn(0x7fedd80fd320 msgr2=0x7fedd80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 shutdown_connections 2024-04-26T21:00:57.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:00:57.472+0000 7feddfe00700 1 -- 172.21.15.204:0/3212053656 wait complete. 2024-04-26T21:00:58.257 DEBUG:teuthology.orchestra.run.smithi094:mon.b> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:00:58.265 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:00:58.269 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:00:58.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:00:58.631 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:58 smithi027 ceph-mon[28085]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:00:58.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:58 smithi027 ceph-mon[28085]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:00:58.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:58 smithi027 ceph-mon[28085]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:00:58.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:58 smithi027 ceph-mon[28085]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c;count:3 2024-04-26T21:00:58.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:58.705 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:00:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:00.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:00 smithi027 ceph-mon[28085]: Deploying daemon mon.c on smithi204 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.241+0000 7f6e6fc00700 1 -- 172.21.15.204:0/792405547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 msgr2=0x7f6e68105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.241+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/792405547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e68105690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009910 tx=0x7f6e5c009c20 comp rx=0 tx=0).stop 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.241+0000 7f6e6fc00700 1 -- 172.21.15.204:0/792405547 shutdown_connections 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.241+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/792405547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e68105690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.241+0000 7f6e6fc00700 1 -- 172.21.15.204:0/792405547 >> 172.21.15.204:0/792405547 conn(0x7f6e68100b40 msgr2=0x7f6e68102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.242+0000 7f6e6fc00700 1 -- 172.21.15.204:0/792405547 shutdown_connections 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.242+0000 7f6e6fc00700 1 -- 172.21.15.204:0/792405547 wait complete. 2024-04-26T21:01:01.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.243+0000 7f6e6fc00700 1 Processor -- start 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.243+0000 7f6e6fc00700 1 -- start start 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.243+0000 7f6e6fc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.243+0000 7f6e6fc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810f3e0 con 0x7f6e681052b0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.244+0000 7f6e6d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.244+0000 7f6e6d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.204:59992/0 (socket says 172.21.15.204:59992) 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.244+0000 7f6e6d200700 1 -- 172.21.15.204:0/1776392806 learned_addr learned my addr 172.21.15.204:0/1776392806 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.244+0000 7f6e6d200700 1 -- 172.21.15.204:0/1776392806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681a0b60 con 0x7f6e681052b0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.245+0000 7f6e6d200700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005660 tx=0x7f6e5c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.245+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c026070 con 0x7f6e681052b0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.245+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e5c00b6f0 con 0x7f6e681052b0 2024-04-26T21:01:01.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.245+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a0e50 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.245+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c0037b0 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.246+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6819a7a0 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.246+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e68106790 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.247+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6e5c029070 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.248+0000 7f6e65600700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 0x7f6e5003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.248+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e5c01e070 con 0x7f6e681052b0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.248+0000 7f6e67e00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 0x7f6e5003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.250+0000 7f6e67e00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 0x7f6e5003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:01.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.264+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6e5c013070 con 0x7f6e681052b0 2024-04-26T21:01:01.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.587+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6e6819bcd0 con 0x7f6e681052b0 2024-04-26T21:01:01.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.588+0000 7f6e65600700 1 -- 172.21.15.204:0/1776392806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6e5c030030 con 0x7f6e681052b0 2024-04-26T21:01:01.589 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:01:01.589 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","modified":"2024-04-26T20:58:46.841103Z","created":"2024-04-26T20:58:46.841103Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:01:01.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 msgr2=0x7f6e5003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 0x7f6e5003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54006e40 comp rx=0 tx=0).stop 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 msgr2=0x7f6e681a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005660 tx=0x7f6e5c005740 comp rx=0 tx=0).stop 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 shutdown_connections 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6e50039650 0x7f6e5003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 --2- 172.21.15.204:0/1776392806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e681052b0 0x7f6e681a0620 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 >> 172.21.15.204:0/1776392806 conn(0x7f6e68100b40 msgr2=0x7f6e68106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 shutdown_connections 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:01.590+0000 7f6e6fc00700 1 -- 172.21.15.204:0/1776392806 wait complete. 2024-04-26T21:01:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-04-26T21:01:01.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:01 smithi027 ceph-mon[28085]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:02.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:02 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/1776392806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:03.496 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:01:03.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:01:03.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:03 smithi027 ceph-mon[28085]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:04.140 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:01:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:04 smithi204 systemd[1]: Starting Ceph mon.c for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:01:05.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 podman[32106]: 2024-04-26 21:01:05.45022879 +0000 UTC m=+0.042237444 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:01:05.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 podman[32106]: 2024-04-26 21:01:05.549815827 +0000 UTC m=+0.141824479 container create 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:01:05.938 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 podman[32106]: 2024-04-26 21:01:05.87204691 +0000 UTC m=+0.464055624 container init 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:01:05.938 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 podman[32106]: 2024-04-26 21:01:05.882665749 +0000 UTC m=+0.474674447 container start 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T21:01:05.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:05 smithi027 ceph-mon[28085]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.192 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:01:06.192 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:01:06.192 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pidfile_write: ignore empty --pid-file 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: load: jerasure load: lrc 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Git sha 0 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: DB SUMMARY 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: DB Session ID: 9KA1EAPP22BVBZEKIWBK 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:01:06.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.env: 0x55c44a152cc0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.info_log: 0x55c47f8afdc0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.statistics: (nil) 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.use_fsync: 0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:01:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.db_log_dir: 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.wal_dir: 2024-04-26T21:01:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:01:06.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.write_buffer_manager: 0x55c47f8810e0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.unordered_write: 0 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:01:06.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.row_cache: None 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.wal_filter: None 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.wal_compression: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:01:06.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:01:06.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_open_files: -1 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Compression algorithms supported: 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:01:06.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kZSTD supported: 0 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.merge_operator: 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_filter: None 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:01:06.201 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c47f8aff80) 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-04-26T21:01:06.202 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x55c47f8b31f0 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-04-26T21:01:06.203 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:01:06.204 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression: NoCompression 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.num_levels: 7 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:01:06.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:01:06.207 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:01:06.208 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:01:06.209 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:01:06.210 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.ttl: 2592000 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:01:06.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5b252264-c924-4942-b01e-b843cb53c093 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165265944828, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165265945789, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714165265, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5b252264-c924-4942-b01e-b843cb53c093", "db_session_id": "9KA1EAPP22BVBZEKIWBK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165265946084, "job": 1, "event": "recovery_finished"} 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c47f9d0000 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: DB pointer 0x55c47f9bc000 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:01:06.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:06.213 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:01:06.214 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x55c47f8b31f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(???) e0 preinit fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 bash[32106]: 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 systemd[1]: Started Ceph mon.c for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T21:01:06.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:01:06.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Added host smithi094 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.217 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Deploying cephadm binary to smithi204 2024-04-26T21:01:06.218 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Added host smithi204 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.219 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c;count:3 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: Deploying daemon mon.c on smithi204 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/1776392806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:06.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:05 smithi204 ceph-mon[32145]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: mon.a calling monitor election 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: mon.c calling monitor election 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: monmap epoch 2 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: last_changed 2024-04-26T21:01:06.000333+0000 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: min_mon_release 19 (squid) 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: election_strategy: 1 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: fsmap 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: mgrmap e13: x(active, since 69s) 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: overall HEALTH_OK 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:11.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: mon.a calling monitor election 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: mon.c calling monitor election 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: monmap epoch 2 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: last_changed 2024-04-26T21:01:06.000333+0000 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: min_mon_release 19 (squid) 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: election_strategy: 1 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: fsmap 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: mgrmap e13: x(active, since 69s) 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: overall HEALTH_OK 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:11.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:11.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:12.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:12 smithi027 ceph-mon[28085]: Deploying daemon mon.b on smithi094 2024-04-26T21:01:12.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:12 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:12 smithi204 ceph-mon[32145]: Deploying daemon mon.b on smithi094 2024-04-26T21:01:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:12 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:12.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.538+0000 7fbd74400700 1 -- 172.21.15.204:0/2686725695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd54006280 msgr2=0x7fbd54008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:12.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.538+0000 7fbd74400700 1 --2- 172.21.15.204:0/2686725695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd54006280 0x7fbd54008710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c07bc20 tx=0x7fbd6000fe90 comp rx=0 tx=0).stop 2024-04-26T21:01:12.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.538+0000 7fbd74400700 1 -- 172.21.15.204:0/2686725695 shutdown_connections 2024-04-26T21:01:12.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.538+0000 7fbd74400700 1 --2- 172.21.15.204:0/2686725695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd54006280 0x7fbd54008710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.541 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.538+0000 7fbd74400700 1 -- 172.21.15.204:0/2686725695 >> 172.21.15.204:0/2686725695 conn(0x7fbd6c078da0 msgr2=0x7fbd6c07b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:12.541 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.539+0000 7fbd74400700 1 -- 172.21.15.204:0/2686725695 shutdown_connections 2024-04-26T21:01:12.541 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.539+0000 7fbd74400700 1 -- 172.21.15.204:0/2686725695 wait complete. 2024-04-26T21:01:12.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.541+0000 7fbd74400700 1 Processor -- start 2024-04-26T21:01:12.542 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.541+0000 7fbd74400700 1 -- start start 2024-04-26T21:01:12.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.542+0000 7fbd74400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:12.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.542+0000 7fbd74400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 0x7fbd6c1a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:12.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.542+0000 7fbd74400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c111340 con 0x7fbd6c19fa30 2024-04-26T21:01:12.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.542+0000 7fbd74400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1111c0 con 0x7fbd6c19f110 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 0x7fbd6c1a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47720/0 (socket says 172.21.15.204:47720) 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 -- 172.21.15.204:0/232982720 learned_addr learned my addr 172.21.15.204:0/232982720 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 -- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 msgr2=0x7fbd6c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 -- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 msgr2=0x7fbd6c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:01:12.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.543+0000 7fbd71a00700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:01:12.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.544+0000 7fbd71000700 1 -- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 msgr2=0x7fbd6c19f4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:01:12.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.544+0000 7fbd71000700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.544+0000 7fbd71000700 1 -- 172.21.15.204:0/232982720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c1a3fa0 con 0x7fbd6c19fa30 2024-04-26T21:01:12.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.545+0000 7fbd71000700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 0x7fbd6c1a39e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800b4d0 tx=0x7fbd5800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:12.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.546+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbd58011630 con 0x7fbd6c19fa30 2024-04-26T21:01:12.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.546+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c1a4290 con 0x7fbd6c19fa30 2024-04-26T21:01:12.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.546+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c1a9c80 con 0x7fbd6c19fa30 2024-04-26T21:01:12.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.546+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd58011c60 con 0x7fbd6c19fa30 2024-04-26T21:01:12.547 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.546+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbd58010c90 con 0x7fbd6c19fa30 2024-04-26T21:01:12.548 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.547+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd54002010 con 0x7fbd6c19fa30 2024-04-26T21:01:12.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.548+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbd58018020 con 0x7fbd6c19fa30 2024-04-26T21:01:12.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.549+0000 7fbd6a000700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 0x7fbd4c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:12.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.549+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd5804c140 con 0x7fbd6c19fa30 2024-04-26T21:01:12.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.550+0000 7fbd71a00700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 0x7fbd4c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:12.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.551+0000 7fbd71a00700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 0x7fbd4c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000fa40 tx=0x7fbd6000f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:12.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.565+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbd5801f020 con 0x7fbd6c19fa30 2024-04-26T21:01:12.912 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.911+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbd540032b0 con 0x7fbd6c19fa30 2024-04-26T21:01:12.913 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.912+0000 7fbd6a000700 1 -- 172.21.15.204:0/232982720 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fbd58018570 con 0x7fbd6c19fa30 2024-04-26T21:01:12.913 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:01:12.913 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","modified":"2024-04-26T21:01:06.000333Z","created":"2024-04-26T20:58:46.841103Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:01:12.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 msgr2=0x7fbd4c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:12.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 0x7fbd4c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000fa40 tx=0x7fbd6000f4a0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 msgr2=0x7fbd6c1a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 0x7fbd6c1a39e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800b4d0 tx=0x7fbd5800b890 comp rx=0 tx=0).stop 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 shutdown_connections 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd4c0393f0 0x7fbd4c03b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd6c19f110 0x7fbd6c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 --2- 172.21.15.204:0/232982720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd6c19fa30 0x7fbd6c1a39e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 >> 172.21.15.204:0/232982720 conn(0x7fbd6c078da0 msgr2=0x7fbd6c079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 shutdown_connections 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:12.915+0000 7fbd74400700 1 -- 172.21.15.204:0/232982720 wait complete. 2024-04-26T21:01:12.917 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-04-26T21:01:13.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:13 smithi204 ceph-mon[32145]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:13.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:13 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/232982720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:13.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:13 smithi027 ceph-mon[28085]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:13.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:13 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/232982720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:01:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:01:13.001+0000 7f077a200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:01:14.638 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:01:14.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:01:15.047 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:01:15.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:15 smithi204 ceph-mon[32145]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:15.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:15 smithi027 ceph-mon[28085]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:16.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 podman[31254]: 2024-04-26 21:01:16.507854005 +0000 UTC m=+0.369341848 container start db99f4544d221f72c8e896d492bdec68ce339d996226c16552fb01ef4eb09384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T21:01:17.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pidfile_write: ignore empty --pid-file 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: load: jerasure load: lrc 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Git sha 0 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: DB SUMMARY 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: DB Session ID: EKIJZDWTMGZIVHDX4MLK 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:01:17.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.env: 0x5603fd80ecc0 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.info_log: 0x56042150bdc0 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.statistics: (nil) 2024-04-26T21:01:17.029 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.use_fsync: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.db_log_dir: 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.wal_dir: 2024-04-26T21:01:17.030 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:01:17.031 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.write_buffer_manager: 0x5604214dd0e0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.unordered_write: 0 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:01:17.032 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.row_cache: None 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.wal_filter: None 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.wal_compression: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:01:17.033 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:01:17.034 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_open_files: -1 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Compression algorithms supported: 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kZSTD supported: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:01:17.035 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.merge_operator: 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_filter: None 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56042150bf80) 2024-04-26T21:01:17.036 INFO:journalctl@ceph.mon.b.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_type: 0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: data_block_index_type: 0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_shortening: 1 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: checksum: 4 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: no_block_cache: 0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache: 0x56042150f1f0 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_options: 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: capacity : 536870912 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: num_shard_bits : 4 2024-04-26T21:01:17.037 INFO:journalctl@ceph.mon.b.smithi094.stdout: strict_capacity_limit : 0 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_compressed: (nil) 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: persistent_cache: (nil) 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_size: 4096 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_size_deviation: 10 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_restart_interval: 16 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_block_restart_interval: 1 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: metadata_block_size: 4096 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: partition_filters: 0 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: use_delta_encoding: 1 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: filter_policy: bloomfilter 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: whole_key_filtering: 1 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: verify_compression: 0 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:01:17.038 INFO:journalctl@ceph.mon.b.smithi094.stdout: format_version: 5 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: enable_index_compression: 1 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_align: 0 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: max_auto_readahead_size: 262144 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: prepopulate_block_cache: 0 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression: NoCompression 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.num_levels: 7 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:01:17.039 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:01:17.040 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:01:17.041 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:01:17.041 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:01:17.041 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:01:17.042 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:01:17.043 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:01:17.044 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.ttl: 2592000 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:01:17.045 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9e285db8-96d7-4d26-a4a9-ca7fe89c3396 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165276565733, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:01:17.046 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165276566864, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714165276, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9e285db8-96d7-4d26-a4a9-ca7fe89c3396", "db_session_id": "EKIJZDWTMGZIVHDX4MLK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714165276567118, "job": 1, "event": "recovery_finished"} 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56042162c000 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: DB pointer 0x560421618000 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** DB Stats ** 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:01:17.047 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:01:17.048 INFO:journalctl@ceph.mon.b.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Block cache BinnedLRUCache@0x56042150f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.049 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: using public_addr v2:172.21.15.94:0/0 -> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: starting mon.b rank -1 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(???) e0 preinit fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 bash[31254]: db99f4544d221f72c8e896d492bdec68ce339d996226c16552fb01ef4eb09384 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: e1 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: legacy client fscid: -1 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-04-26T21:01:17.050 INFO:journalctl@ceph.mon.b.smithi094.stdout: No filesystems configured 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 systemd[1]: Started Ceph mon.b for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Added host smithi094 2024-04-26T21:01:17.051 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:17.052 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Deploying cephadm binary to smithi204 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Added host smithi204 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.053 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2558168279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.054 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c;count:3 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.055 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Deploying daemon mon.c on smithi204 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/1776392806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.a calling monitor election 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.056 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.c calling monitor election 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: monmap epoch 2 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: last_changed 2024-04-26T21:01:06.000333+0000 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: min_mon_release 19 (squid) 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: election_strategy: 1 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:17.057 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: fsmap 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mgrmap e13: x(active, since 69s) 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: overall HEALTH_OK 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: Deploying daemon mon.b on smithi094 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/232982720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:17.058 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:17.059 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:16 smithi094 ceph-mon[31273]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-26T21:01:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: mon.a calling monitor election 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: mon.c calling monitor election 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: mon.b calling monitor election 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:21.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: monmap epoch 3 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: last_changed 2024-04-26T21:01:16.622835+0000 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: min_mon_release 19 (squid) 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: election_strategy: 1 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: fsmap 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: mgrmap e13: x(active, since 79s) 2024-04-26T21:01:21.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: overall HEALTH_OK 2024-04-26T21:01:21.988 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:21.988 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: mon.a calling monitor election 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: mon.c calling monitor election 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: mon.b calling monitor election 2024-04-26T21:01:22.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: monmap epoch 3 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: last_changed 2024-04-26T21:01:16.622835+0000 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: min_mon_release 19 (squid) 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: election_strategy: 1 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-26T21:01:22.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: fsmap 2024-04-26T21:01:22.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:22.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: mgrmap e13: x(active, since 79s) 2024-04-26T21:01:22.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: overall HEALTH_OK 2024-04-26T21:01:22.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: mon.a calling monitor election 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: mon.c calling monitor election 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: mon.b calling monitor election 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: monmap epoch 3 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: last_changed 2024-04-26T21:01:16.622835+0000 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: created 2024-04-26T20:58:46.841103+0000 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: min_mon_release 19 (squid) 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: election_strategy: 1 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-04-26T21:01:22.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: fsmap 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: mgrmap e13: x(active, since 79s) 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: overall HEALTH_OK 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.965+0000 7f9cf5400700 1 -- 172.21.15.204:0/705466367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cd4005c90 msgr2=0x7f9cd40080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:22.967 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.965+0000 7f9cf5400700 1 --2- 172.21.15.204:0/705466367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cd4005c90 0x7f9cd40080c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0104390 tx=0x7f9ce000ff20 comp rx=0 tx=0).stop 2024-04-26T21:01:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.965+0000 7f9cf5400700 1 -- 172.21.15.204:0/705466367 shutdown_connections 2024-04-26T21:01:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.965+0000 7f9cf5400700 1 --2- 172.21.15.204:0/705466367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cd4005c90 0x7f9cd40080c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.965+0000 7f9cf5400700 1 -- 172.21.15.204:0/705466367 >> 172.21.15.204:0/705466367 conn(0x7f9cf00fe9f0 msgr2=0x7f9cf0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.966+0000 7f9cf5400700 1 -- 172.21.15.204:0/705466367 shutdown_connections 2024-04-26T21:01:22.968 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.966+0000 7f9cf5400700 1 -- 172.21.15.204:0/705466367 wait complete. 2024-04-26T21:01:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.967+0000 7f9cf5400700 1 Processor -- start 2024-04-26T21:01:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.967+0000 7f9cf5400700 1 -- start start 2024-04-26T21:01:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:22.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 0x7f9cf019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 0x7f9cf01a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf010ace0 con 0x7f9cf019b2c0 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf010ab60 con 0x7f9cf019fb90 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cf5400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf010ae60 con 0x7f9cd4005c90 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cee800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cee800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53236/0 (socket says 172.21.15.204:53236) 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cee800700 1 -- 172.21.15.204:0/2053681670 learned_addr learned my addr 172.21.15.204:0/2053681670 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:01:22.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.968+0000 7f9cede00700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 0x7f9cf019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:22.971 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cef200700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 0x7f9cf01a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:22.971 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 -- 172.21.15.204:0/2053681670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 msgr2=0x7f9cf01a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 0x7f9cf01a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 -- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 msgr2=0x7f9cf019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 0x7f9cf019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 -- 172.21.15.204:0/2053681670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf01a7900 con 0x7f9cd4005c90 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cef200700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 0x7f9cf01a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cede00700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 0x7f9cf019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.969+0000 7f9cee800700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0014ff0 tx=0x7f9ce00046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.970+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce002b020 con 0x7f9cd4005c90 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.970+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ce00055f0 con 0x7f9cd4005c90 2024-04-26T21:01:22.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.970+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf01a7ad0 con 0x7f9cd4005c90 2024-04-26T21:01:22.973 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.970+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0014770 con 0x7f9cd4005c90 2024-04-26T21:01:22.973 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.970+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01a8060 con 0x7f9cd4005c90 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.971+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9ce002e020 con 0x7f9cd4005c90 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.971+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf0064a80 con 0x7f9cd4005c90 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.972+0000 7f9ce7400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 0x7f9cd003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.972+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ce004f990 con 0x7f9cd4005c90 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.972+0000 7f9cede00700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 0x7f9cd003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:22.974 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.973+0000 7f9cede00700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 0x7f9cd003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009ec0 tx=0x7f9cdc009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:22.979 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:22.978+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9ce0035020 con 0x7f9cd4005c90 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:22.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:23.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:23.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:01:23.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.302+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9cf002e490 con 0x7f9cd4005c90 2024-04-26T21:01:23.304 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.302+0000 7f9ce7400700 1 -- 172.21.15.204:0/2053681670 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f9ce0026e00 con 0x7f9cd4005c90 2024-04-26T21:01:23.304 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-26T21:01:23.304 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","modified":"2024-04-26T21:01:16.622835Z","created":"2024-04-26T20:58:46.841103Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.305+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 msgr2=0x7f9cd003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.305+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 0x7f9cd003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009ec0 tx=0x7f9cdc009210 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.305+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 msgr2=0x7f9cf019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.305+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0014ff0 tx=0x7f9ce00046d0 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 shutdown_connections 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9cd0039ae0 0x7f9cd003bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf019fb90 0x7f9cf01a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd4005c90 0x7f9cf019ad80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 --2- 172.21.15.204:0/2053681670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cf019b2c0 0x7f9cf019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 >> 172.21.15.204:0/2053681670 conn(0x7f9cf00fe9f0 msgr2=0x7f9cf00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:23.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 shutdown_connections 2024-04-26T21:01:23.308 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:23.306+0000 7f9cf5400700 1 -- 172.21.15.204:0/2053681670 wait complete. 2024-04-26T21:01:23.308 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-04-26T21:01:23.962 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:23 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/2053681670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:24.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:23 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/2053681670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:24.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:23 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/2053681670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:01:24.678 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:01:24.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:01:25.112 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:01:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:01:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:24 smithi027 ceph-mon[28085]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:01:25.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:25.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:01:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:24 smithi204 ceph-mon[32145]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:01:25.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:25.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:01:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:24 smithi094 ceph-mon[31273]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.341 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:26.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.191+0000 7f00cee00700 1 -- 172.21.15.27:0/898883239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c8103f00 msgr2=0x7f00c81042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.191+0000 7f00cee00700 1 --2- 172.21.15.27:0/898883239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c8103f00 0x7f00c81042e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009910 tx=0x7f00bc009c20 comp rx=0 tx=0).stop 2024-04-26T21:01:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.191+0000 7f00cee00700 1 -- 172.21.15.27:0/898883239 shutdown_connections 2024-04-26T21:01:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.191+0000 7f00cee00700 1 --2- 172.21.15.27:0/898883239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c8103f00 0x7f00c81042e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.191+0000 7f00cee00700 1 -- 172.21.15.27:0/898883239 >> 172.21.15.27:0/898883239 conn(0x7f00c80fd6b0 msgr2=0x7f00c80ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.192+0000 7f00cee00700 1 -- 172.21.15.27:0/898883239 shutdown_connections 2024-04-26T21:01:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.192+0000 7f00cee00700 1 -- 172.21.15.27:0/898883239 wait complete. 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.193+0000 7f00cee00700 1 Processor -- start 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.193+0000 7f00cee00700 1 -- start start 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.193+0000 7f00cee00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c8103f00 0x7f00c81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cee00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c819ae60 0x7f00c819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00c810d0b0 con 0x7f00c81a0ca0 2024-04-26T21:01:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cee00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00c810cf30 con 0x7f00c8103f00 2024-04-26T21:01:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cee00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00c810d230 con 0x7f00c819ae60 2024-04-26T21:01:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cd000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cd000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44284/0 (socket says 172.21.15.27:44284) 2024-04-26T21:01:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cd000700 1 -- 172.21.15.27:0/185177997 learned_addr learned my addr 172.21.15.27:0/185177997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.194+0000 7f00cda00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c8103f00 0x7f00c81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 -- 172.21.15.27:0/185177997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c819ae60 msgr2=0x7f00c819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c819ae60 0x7f00c819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 -- 172.21.15.27:0/185177997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c8103f00 msgr2=0x7f00c81a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c8103f00 0x7f00c81a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 -- 172.21.15.27:0/185177997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c81a7630 con 0x7f00c81a0ca0 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.195+0000 7f00cd000700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f00b400b780 tx=0x7f00b400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.196+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004230 con 0x7f00c81a0ca0 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.196+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00b40043d0 con 0x7f00c81a0ca0 2024-04-26T21:01:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.196+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4011ac0 con 0x7f00c81a0ca0 2024-04-26T21:01:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.196+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c81a7860 con 0x7f00c81a0ca0 2024-04-26T21:01:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.196+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81a7df0 con 0x7f00c81a0ca0 2024-04-26T21:01:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.197+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c8064a80 con 0x7f00c81a0ca0 2024-04-26T21:01:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.197+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f00b40036a0 con 0x7f00c81a0ca0 2024-04-26T21:01:27.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.198+0000 7f00c6000700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 0x7f00b003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:27.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.198+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f00b404c560 con 0x7f00c81a0ca0 2024-04-26T21:01:27.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.198+0000 7f00cda00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 0x7f00b003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.199+0000 7f00cda00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 0x7f00b003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005920 tx=0x7f00bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:27.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.204+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00b4020b60 con 0x7f00c81a0ca0 2024-04-26T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.448+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f00c802e650 con 0x7f00c81a0ca0 2024-04-26T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.449+0000 7f00c6000700 1 -- 172.21.15.27:0/185177997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f00b4010380 con 0x7f00c81a0ca0 2024-04-26T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-04-26T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-04-26T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 msgr2=0x7f00b003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 0x7f00b003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005920 tx=0x7f00bc01a040 comp rx=0 tx=0).stop 2024-04-26T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 msgr2=0x7f00c819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f00b400b780 tx=0x7f00b400bb40 comp rx=0 tx=0).stop 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 shutdown_connections 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f00b0039ae0 0x7f00b003bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c819ae60 0x7f00c819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00c8103f00 0x7f00c81a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 --2- 172.21.15.27:0/185177997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00c81a0ca0 0x7f00c819a920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 >> 172.21.15.27:0/185177997 conn(0x7f00c80fd6b0 msgr2=0x7f00c8109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 shutdown_connections 2024-04-26T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:27.453+0000 7f00cee00700 1 -- 172.21.15.27:0/185177997 wait complete. 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:27.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:27 smithi027 ceph-mon[28085]: Reconfiguring daemon mon.a on smithi027 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:27.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:27 smithi204 ceph-mon[32145]: Reconfiguring daemon mon.a on smithi027 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:27 smithi094 ceph-mon[31273]: Reconfiguring daemon mon.a on smithi027 2024-04-26T21:01:28.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:28 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/185177997' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:28.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:28 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/185177997' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:28.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:28 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/185177997' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:29.180 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:01:29.180 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:01:29.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:01:29.275 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:01:29.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:29.416 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T21:01:29.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:01:29.491 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T21:01:29.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:29.594 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:01:29.594 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:29.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:29.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:29.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:29.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:29.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:29.695 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:01:29.695 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:29.825 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-04-26T21:01:29.825 INFO:tasks.cephadm:Adding mgr.y on smithi094 2024-04-26T21:01:29.825 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-04-26T21:01:29.825 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi027=x;smithi094=y;smithi204=z' 2024-04-26T21:01:30.307 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:01:30.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:30 smithi094 ceph-mon[31273]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:01:30.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:30 smithi094 ceph-mon[31273]: Reconfiguring daemon mon.b on smithi094 2024-04-26T21:01:30.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:30 smithi204 ceph-mon[32145]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:01:30.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:30 smithi204 ceph-mon[32145]: Reconfiguring daemon mon.b on smithi094 2024-04-26T21:01:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:30 smithi027 ceph-mon[28085]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:01:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:30 smithi027 ceph-mon[28085]: Reconfiguring daemon mon.b on smithi094 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:31.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:31.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:01:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:01:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:32.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:32 smithi204 ceph-mon[32145]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:01:32.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:32 smithi204 ceph-mon[32145]: Reconfiguring daemon mon.c on smithi204 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 -- 172.21.15.204:0/2781816699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25901051a0 msgr2=0x7f25901055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 --2- 172.21.15.204:0/2781816699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25901051a0 0x7f25901055c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2580009910 tx=0x7f2580009c20 comp rx=0 tx=0).stop 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 -- 172.21.15.204:0/2781816699 shutdown_connections 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 --2- 172.21.15.204:0/2781816699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25901051a0 0x7f25901055c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 -- 172.21.15.204:0/2781816699 >> 172.21.15.204:0/2781816699 conn(0x7f2590100760 msgr2=0x7f2590102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.589+0000 7f2596c00700 1 -- 172.21.15.204:0/2781816699 shutdown_connections 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.590+0000 7f2596c00700 1 -- 172.21.15.204:0/2781816699 wait complete. 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 Processor -- start 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 -- start start 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25901051a0 0x7f259019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:32.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 0x7f259019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f259010af90 con 0x7f259019ad40 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f259010ae10 con 0x7f25901051a0 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.591+0000 7f2596c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f259010b110 con 0x7f259019b1c0 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:42498/0 (socket says 172.21.15.204:42498) 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 -- 172.21.15.204:0/3116282221 learned_addr learned my addr 172.21.15.204:0/3116282221 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f258f400700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 0x7f259019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f258fe00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25901051a0 0x7f259019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 -- 172.21.15.204:0/3116282221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25901051a0 msgr2=0x7f259019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25901051a0 0x7f259019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 -- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 msgr2=0x7f259019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:32.725 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 0x7f259019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f2594c00700 1 -- 172.21.15.204:0/3116282221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25901a2c10 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.592+0000 7f258f400700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 0x7f259019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f2594c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f258800ba50 tx=0x7f258800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588019070 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2588014440 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25880075b0 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25901a2f00 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.593+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a3480 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.594+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2590064a80 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.594+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f258801c070 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.595+0000 7f258cc00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 0x7f257403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.595+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f258804bf10 con 0x7f259019b1c0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.596+0000 7f258fe00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 0x7f257403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:32.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.597+0000 7f258fe00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 0x7f257403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2580005920 tx=0x7f2580005190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:32.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.601+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f258801b2c0 con 0x7f259019b1c0 2024-04-26T21:01:32.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:32 smithi027 ceph-mon[28085]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:01:32.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:32 smithi027 ceph-mon[28085]: Reconfiguring daemon mon.c on smithi204 2024-04-26T21:01:32.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:32 smithi094 ceph-mon[31273]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:01:32.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:32 smithi094 ceph-mon[31273]: Reconfiguring daemon mon.c on smithi204 2024-04-26T21:01:32.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.847+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi094=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f2590063860 con 0x7f2574039ae0 2024-04-26T21:01:32.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.862+0000 7f258cc00700 1 -- 172.21.15.204:0/3116282221 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2590063860 con 0x7f2574039ae0 2024-04-26T21:01:32.863 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-04-26T21:01:32.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.864+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 msgr2=0x7f257403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:32.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.864+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 0x7f257403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2580005920 tx=0x7f2580005190 comp rx=0 tx=0).stop 2024-04-26T21:01:32.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 msgr2=0x7f25901a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f258800ba50 tx=0x7f258800be10 comp rx=0 tx=0).stop 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 shutdown_connections 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2574039ae0 0x7f257403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f259019b1c0 0x7f25901a2340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25901051a0 0x7f259019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 --2- 172.21.15.204:0/3116282221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259019ad40 0x7f259019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 >> 172.21.15.204:0/3116282221 conn(0x7f2590100760 msgr2=0x7f2590101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 shutdown_connections 2024-04-26T21:01:32.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:01:32.865+0000 7f2596c00700 1 -- 172.21.15.204:0/3116282221 wait complete. 2024-04-26T21:01:33.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:33 smithi027 ceph-mon[28085]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:33.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:33.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:33 smithi204 ceph-mon[32145]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:33.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:33.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:33 smithi094 ceph-mon[31273]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:33.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:33.979 DEBUG:teuthology.orchestra.run.smithi094:mgr.y> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:01:33.987 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:01:33.993 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:01:33.993 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:01:33.993 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:01:34.054 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:01:34.054 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-04-26T21:01:34.108 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 21:00:18.719844747 +0000 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 20:54:31.684023816 +0000 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 20:54:31.684023816 +0000 2024-04-26T21:01:34.109 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 20:54:31.684023816 +0000 2024-04-26T21:01:34.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:01:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T21:01:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T21:01:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000431101 s, 1.2 MB/s 2024-04-26T21:01:34.198 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:01:34.256 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-04-26T21:01:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:01:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:34.309 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 21:00:18.776843289 +0000 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 20:54:32.281008090 +0000 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 20:54:32.281008090 +0000 2024-04-26T21:01:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 20:54:32.281008090 +0000 2024-04-26T21:01:34.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:01:34.395 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T21:01:34.395 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T21:01:34.396 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000365118 s, 1.4 MB/s 2024-04-26T21:01:34.400 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:01:34.460 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 21:00:18.849841422 +0000 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 20:54:32.908991548 +0000 2024-04-26T21:01:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 20:54:32.908991548 +0000 2024-04-26T21:01:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 20:54:32.908991548 +0000 2024-04-26T21:01:34.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:01:34.602 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T21:01:34.603 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T21:01:34.603 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000379548 s, 1.3 MB/s 2024-04-26T21:01:34.606 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:01:34.663 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-04-26T21:01:34.716 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:01:34.716 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:34.716 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 21:00:18.900840117 +0000 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 20:54:33.549974663 +0000 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 20:54:33.549974663 +0000 2024-04-26T21:01:34.717 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-26 20:54:33.549974663 +0000 2024-04-26T21:01:34.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi094=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: Saving service mgr spec with placement smithi027=x;smithi094=y;smithi204=z;count:3 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.789 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:34 smithi027 ceph-mon[28085]: Deploying daemon mgr.z on smithi204 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi094=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: Saving service mgr spec with placement smithi027=x;smithi094=y;smithi204=z;count:3 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:34.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:34 smithi204 ceph-mon[32145]: Deploying daemon mgr.z on smithi204 2024-04-26T21:01:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-26T21:01:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-26T21:01:34.798 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000392943 s, 1.3 MB/s 2024-04-26T21:01:34.801 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:01:34.859 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T21:01:34.859 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:01:34.920 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:01:34.920 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 21:01:15.730215441 +0000 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 20:54:20.257222701 +0000 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 20:54:20.257222701 +0000 2024-04-26T21:01:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 20:54:20.257222701 +0000 2024-04-26T21:01:34.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi094=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: Saving service mgr spec with placement smithi027=x;smithi094=y;smithi204=z;count:3 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:35.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:35.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:34 smithi094 ceph-mon[31273]: Deploying daemon mgr.z on smithi204 2024-04-26T21:01:35.057 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T21:01:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T21:01:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000271652 s, 1.9 MB/s 2024-04-26T21:01:35.062 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:01:35.120 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-26T21:01:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 21:01:15.731215425 +0000 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 20:54:20.868212854 +0000 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 20:54:20.868212854 +0000 2024-04-26T21:01:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 20:54:20.868212854 +0000 2024-04-26T21:01:35.170 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:01:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T21:01:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T21:01:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000257622 s, 2.0 MB/s 2024-04-26T21:01:35.248 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:01:35.306 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 21:01:15.732215409 +0000 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 20:54:21.444203571 +0000 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 20:54:21.444203571 +0000 2024-04-26T21:01:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 20:54:21.444203571 +0000 2024-04-26T21:01:35.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:01:35.433 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T21:01:35.475 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T21:01:35.476 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000267181 s, 1.9 MB/s 2024-04-26T21:01:35.477 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:01:35.537 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-26 21:01:15.733215393 +0000 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-26 20:54:22.039193982 +0000 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-26 20:54:22.039193982 +0000 2024-04-26T21:01:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-26 20:54:22.039193982 +0000 2024-04-26T21:01:35.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:01:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-26T21:01:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-26T21:01:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000276465 s, 1.9 MB/s 2024-04-26T21:01:35.671 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:01:35.724 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-26T21:01:35.725 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:01:35.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:35 smithi094 ceph-mon[31273]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:35.782 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:01:35.783 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-04-26T21:01:35.835 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:01:35.835 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:01:04.644792659 +0000 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 20:54:21.562836821 +0000 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 20:54:21.562836821 +0000 2024-04-26T21:01:35.836 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 20:54:21.562836821 +0000 2024-04-26T21:01:35.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:01:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:35 smithi204 ceph-mon[32145]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:35.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:35 smithi027 ceph-mon[28085]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:36.037 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:01:36.037 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:01:36.037 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000337866 s, 1.5 MB/s 2024-04-26T21:01:36.042 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:01:36.090 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-04-26T21:01:36.181 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:01:35.933392648 +0000 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 20:54:22.174828944 +0000 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 20:54:22.174828944 +0000 2024-04-26T21:01:36.182 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 20:54:22.174828944 +0000 2024-04-26T21:01:36.182 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:01:36.348 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:01:36.348 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:01:36.348 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000347563 s, 1.5 MB/s 2024-04-26T21:01:36.351 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:01:36.400 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-04-26T21:01:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:01:36.252388519 +0000 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 20:54:22.767821312 +0000 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 20:54:22.767821312 +0000 2024-04-26T21:01:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 20:54:22.767821312 +0000 2024-04-26T21:01:36.442 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:01:36.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:36 smithi204 systemd[1]: Starting Ceph mgr.z for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:01:36.543 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:01:36.543 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:01:36.543 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000331554 s, 1.5 MB/s 2024-04-26T21:01:36.546 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:01:36.633 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-26 21:01:36.253388506 +0000 2024-04-26T21:01:36.673 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-26 20:54:23.383813383 +0000 2024-04-26T21:01:36.674 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-26 20:54:23.383813383 +0000 2024-04-26T21:01:36.674 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-04-26 20:54:23.383813383 +0000 2024-04-26T21:01:36.674 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:01:36.779 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-26T21:01:36.779 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-26T21:01:36.779 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.0003761 s, 1.4 MB/s 2024-04-26T21:01:36.782 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:01:36.824 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-04-26T21:01:36.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:01:36.893 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:36 smithi204 podman[33807]: 2024-04-26 21:01:36.635479269 +0000 UTC m=+0.038352163 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:01:36.894 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:36 smithi204 podman[33807]: 2024-04-26 21:01:36.746243976 +0000 UTC m=+0.149116867 container create 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:01:37.232 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:36 smithi204 podman[33807]: 2024-04-26 21:01:36.966987641 +0000 UTC m=+0.369860578 container init 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:01:37.232 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:36 smithi204 podman[33807]: 2024-04-26 21:01:36.977411852 +0000 UTC m=+0.380284818 container start 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:01:37.232 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mgr[33898]: -- 172.21.15.204:0/2915322815 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55908ea9e5a0 con 0x55908ea73000 2024-04-26T21:01:37.232 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 bash[33807]: 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b 2024-04-26T21:01:37.233 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 systemd[1]: Started Ceph mgr.z for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:01:37.281 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:01:37.486 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:37.230+0000 7ff5cf8b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:01:37.487 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:37.484+0000 7ff5cf8b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:01:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:37.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:37.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:37.614+0000 7ff5cf8b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:37.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:37.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:37.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:38.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:38.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:38.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:38.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:38.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:38.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:01:38.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:38.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:38.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:38.014+0000 7ff5cf8b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:01:38.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:38.125+0000 7ff5cf8b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:01:38.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:38 smithi027 ceph-mon[28085]: Deploying daemon mgr.y on smithi094 2024-04-26T21:01:39.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:38 smithi204 ceph-mon[32145]: Deploying daemon mgr.y on smithi094 2024-04-26T21:01:39.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:38 smithi094 ceph-mon[31273]: Deploying daemon mgr.y on smithi094 2024-04-26T21:01:39.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:39 smithi027 ceph-mon[28085]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:39.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:39 smithi094 ceph-mon[31273]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:39.916 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:39 smithi204 ceph-mon[32145]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:39.916 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:39.689+0000 7ff5cf8b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:01:40.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:39.914+0000 7ff5cf8b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:01:40.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:40.131+0000 7ff5cf8b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:01:40.409 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:01:40.409 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:01:40.409 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T21:01:40.409 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T21:01:40.410 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0388656 s, 270 MB/s 2024-04-26T21:01:40.410 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T21:01:40.417 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 systemd[1]: Starting Ceph mgr.y for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:01:40.677 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 podman[32645]: 2024-04-26 21:01:40.32297446 +0000 UTC m=+0.040054815 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:01:40.677 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 podman[32645]: 2024-04-26 21:01:40.437285615 +0000 UTC m=+0.154366000 container create c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:01:40.677 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 podman[32645]: 2024-04-26 21:01:40.665988116 +0000 UTC m=+0.383068511 container init c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:01:40.677 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 podman[32645]: 2024-04-26 21:01:40.676007405 +0000 UTC m=+0.393087776 container start c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:01:40.882 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:40.505+0000 7ff5cf8b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:01:40.882 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:40.626+0000 7ff5cf8b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:01:41.025 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 ceph-mgr[32663]: -- 172.21.15.94:0/359739203 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55962df405a0 con 0x55962df14800 2024-04-26T21:01:41.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 bash[32645]: c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 2024-04-26T21:01:41.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 systemd[1]: Started Ceph mgr.y for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:01:41.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:40.922+0000 7f75343f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:01:41.136 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:40.880+0000 7ff5cf8b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:01:41.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-04-26T21:01:41.497 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:41.135+0000 7ff5cf8b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:01:41.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:41.183+0000 7f75343f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:01:41.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:41.321+0000 7f75343f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:41.793 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:41.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:42.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:41.740+0000 7f75343f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:01:42.027 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:41.856+0000 7f75343f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:01:42.251 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:41.984+0000 7ff5cf8b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:01:42.986 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:42.580+0000 7ff5cf8b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:01:42.987 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:42.717+0000 7ff5cf8b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:01:43.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:42 smithi027 ceph-mon[28085]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:43.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:43.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:42 smithi204 ceph-mon[32145]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:43.251 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:43.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:42.990+0000 7ff5cf8b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:01:43.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:42 smithi094 ceph-mon[31273]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:43.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:43.775 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:43.496+0000 7f75343f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:01:43.776 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:43.732+0000 7f75343f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:01:44.002 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:43.669+0000 7ff5cf8b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:01:44.002 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:43.792+0000 7ff5cf8b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:01:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.009+0000 7fc9b8200700 1 -- 172.21.15.27:0/2724775581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b0107270 msgr2=0x7fc9b010db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.009+0000 7fc9b8200700 1 --2- 172.21.15.27:0/2724775581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b0107270 0x7fc9b010db20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80099c0 tx=0x7fc9a8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 -- 172.21.15.27:0/2724775581 shutdown_connections 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 --2- 172.21.15.27:0/2724775581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b0107270 0x7fc9b010db20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 --2- 172.21.15.27:0/2724775581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01056d0 0x7fc9b0105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 --2- 172.21.15.27:0/2724775581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b01068d0 0x7fc9b0106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 -- 172.21.15.27:0/2724775581 >> 172.21.15.27:0/2724775581 conn(0x7fc9b0100c70 msgr2=0x7fc9b01030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 -- 172.21.15.27:0/2724775581 shutdown_connections 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.010+0000 7fc9b8200700 1 -- 172.21.15.27:0/2724775581 wait complete. 2024-04-26T21:01:44.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.011+0000 7fc9b8200700 1 Processor -- start 2024-04-26T21:01:44.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 -- start start 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01068d0 0x7fc9b019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 0x7fc9b01a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b01110c0 con 0x7fc9b01056d0 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0110f40 con 0x7fc9b0107270 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b8200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0111240 con 0x7fc9b01068d0 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b5800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b5800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43268/0 (socket says 172.21.15.27:43268) 2024-04-26T21:01:44.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.012+0000 7fc9b5800700 1 -- 172.21.15.27:0/1076805429 learned_addr learned my addr 172.21.15.27:0/1076805429 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b4e00700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01068d0 0x7fc9b019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b6200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 0x7fc9b01a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 -- 172.21.15.27:0/1076805429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01068d0 msgr2=0x7fc9b019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01068d0 0x7fc9b019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 -- 172.21.15.27:0/1076805429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 msgr2=0x7fc9b01a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 0x7fc9b01a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 -- 172.21.15.27:0/1076805429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01a3390 con 0x7fc9b01056d0 2024-04-26T21:01:44.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b6200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 0x7fc9b01a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.013+0000 7fc9b5800700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009f40 tx=0x7fc9a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.014+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00092e0 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.014+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a0003a00 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.014+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00044e0 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.014+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b01a3620 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.014+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b01a3be0 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.015+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc9a001b030 con 0x7fc9b01056d0 2024-04-26T21:01:44.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.015+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b006a110 con 0x7fc9b01056d0 2024-04-26T21:01:44.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.016+0000 7fc9ae000700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 0x7fc99403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:01:44.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.016+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9a004ca30 con 0x7fc9b01056d0 2024-04-26T21:01:44.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.016+0000 7fc9b4e00700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 0x7fc99403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:01:44.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.017+0000 7fc9b4e00700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 0x7fc99403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a10 tx=0x7fc99c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:01:44.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.022+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc9a0022020 con 0x7fc9b01056d0 2024-04-26T21:01:44.238 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:43.954+0000 7f75343f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:01:44.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:44.267+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc9b0063860 con 0x7fc994039890 2024-04-26T21:01:44.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.423 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:44.115+0000 7ff5cf8b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:01:44.423 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:44.294+0000 7ff5cf8b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:01:44.423 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:44.420+0000 7ff5cf8b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:01:44.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:44.364+0000 7f75343f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:01:44.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:44.484+0000 7f75343f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:01:44.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:44 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:44 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:44 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:44 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:44.751 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:44.573+0000 7ff5cf8b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:01:44.988 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:44.736+0000 7f75343f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:01:44.989 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:44.986+0000 7f75343f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:01:45.251 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:44.949+0000 7ff5cf8b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:01:45.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:45.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:45.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:45.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:45.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:01:45.704 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:01:45.704 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:45.988 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:45.700+0000 7ff5cf8b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:01:45.988 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:01:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:01:45.986+0000 7ff5cf8b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:01:46.025 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:45.767+0000 7f75343f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:01:46.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:46.293+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fc9a0012050 con 0x7fc9b01056d0 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: Standby manager daemon z started 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:01:46.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:46 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:46.701 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:46.313+0000 7f75343f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:01:46.701 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:46.441+0000 7f75343f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:01:46.701 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:46.699+0000 7f75343f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:01:46.701 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: Standby manager daemon z started 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:01:46.702 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:46 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: Reconfiguring daemon mgr.x on smithi027 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: Standby manager daemon z started 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:01:46.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:46 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/2836202973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:47.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:47 smithi027 ceph-mon[28085]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:47 smithi027 ceph-mon[28085]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-26T21:01:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:01:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:47 smithi204 ceph-mon[32145]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:47 smithi204 ceph-mon[32145]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-26T21:01:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:01:47.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-mon[31273]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-mon[31273]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-26T21:01:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:01:47.776 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:47.341+0000 7f75343f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:01:47.776 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:47.463+0000 7f75343f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:01:48.083 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:47.778+0000 7f75343f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:01:48.083 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:47.955+0000 7f75343f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:01:48.083 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:48.081+0000 7f75343f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:01:48.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:48.234+0000 7f75343f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:01:48.948 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:48.598+0000 7f75343f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:01:49.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:49.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:49.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:49.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:49.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:01:49.629 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:49.342+0000 7f75343f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:01:49.951 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:01:49.627+0000 7f75343f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:01:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:01:49.981+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fc9a00133e0 con 0x7fc9b01056d0 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: Standby manager daemon y started 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]: dispatch 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]': finished 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:01:50.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: Standby manager daemon y started 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]: dispatch 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]': finished 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:01:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: Standby manager daemon y started 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/4032520797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]: dispatch 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4287277538' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa98cfe0-26a3-4f94-bc02-6c01411e6ec3"}]': finished 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:01:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:01:51.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:51 smithi094 ceph-mon[31273]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:51.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:51 smithi094 ceph-mon[31273]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-26T21:01:51.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:51 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:01:51.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3519964218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:01:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:51 smithi027 ceph-mon[28085]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:51 smithi027 ceph-mon[28085]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-26T21:01:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:51 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:01:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3519964218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:01:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:51 smithi204 ceph-mon[32145]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:51 smithi204 ceph-mon[32145]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-26T21:01:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:51 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:01:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3519964218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:01:53.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:53 smithi204 ceph-mon[32145]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:53.279 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:53 smithi094 ceph-mon[31273]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:53.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:53 smithi027 ceph-mon[28085]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:55.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:55 smithi027 ceph-mon[28085]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:55.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:55 smithi204 ceph-mon[32145]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:55.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:55 smithi094 ceph-mon[31273]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:57.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:57 smithi027 ceph-mon[28085]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:57.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:57 smithi204 ceph-mon[32145]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:57.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:57 smithi094 ceph-mon[31273]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:59.348 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:01:59 smithi027 ceph-mon[28085]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:01:59 smithi204 ceph-mon[32145]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:01:59.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:01:59 smithi094 ceph-mon[31273]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:00.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:02:00.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:00.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:00 smithi027 ceph-mon[28085]: Deploying daemon osd.0 on smithi027 2024-04-26T21:02:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:02:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:00 smithi204 ceph-mon[32145]: Deploying daemon osd.0 on smithi027 2024-04-26T21:02:00.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:02:00.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:00.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:00 smithi094 ceph-mon[31273]: Deploying daemon osd.0 on smithi027 2024-04-26T21:02:01.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:01 smithi094 ceph-mon[31273]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:01.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:01 smithi027 ceph-mon[28085]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:01 smithi204 ceph-mon[32145]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:03 smithi027 ceph-mon[28085]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:03 smithi204 ceph-mon[32145]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:03.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:03 smithi094 ceph-mon[31273]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:05.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:05 smithi027 ceph-mon[28085]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:05 smithi204 ceph-mon[32145]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:05.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:05 smithi094 ceph-mon[31273]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:07.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:07 smithi027 ceph-mon[28085]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:07.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:07 smithi204 ceph-mon[32145]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:07.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:07 smithi094 ceph-mon[31273]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:09.856 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:09 smithi027 ceph-mon[28085]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:09.856 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:09.856 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:09.856 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:09 smithi204 ceph-mon[32145]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:10.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:09 smithi094 ceph-mon[31273]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:11.844 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:11 smithi027 ceph-mon[28085]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:11 smithi204 ceph-mon[32145]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:12.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:11 smithi094 ceph-mon[31273]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:13.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:13.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:13.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:13.450 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.450 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:13 smithi027 ceph-mon[28085]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:13 smithi204 ceph-mon[32145]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:13.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:13 smithi094 ceph-mon[31273]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:02:13.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.531+0000 7fc9ae000700 1 -- 172.21.15.27:0/1076805429 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc9b0063860 con 0x7fc994039890 2024-04-26T21:02:13.533 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.535+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 msgr2=0x7fc99403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.535+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 0x7fc99403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009a10 tx=0x7fc99c009190 comp rx=0 tx=0).stop 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.535+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 msgr2=0x7fc9b019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.535+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009f40 tx=0x7fc9a000c3a0 comp rx=0 tx=0).stop 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 shutdown_connections 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc994039890 0x7fc99403bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9b0107270 0x7fc9b01a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9b01056d0 0x7fc9b019b730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 --2- 172.21.15.27:0/1076805429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b01068d0 0x7fc9b019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 >> 172.21.15.27:0/1076805429 conn(0x7fc9b0100c70 msgr2=0x7fc9b0109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:02:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 shutdown_connections 2024-04-26T21:02:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:02:13.536+0000 7fc9b8200700 1 -- 172.21.15.27:0/1076805429 wait complete. 2024-04-26T21:02:14.311 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:02:14.317 INFO:tasks.cephadm:Deploying osd.1 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-26T21:02:14.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:02:14.751 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:02:14.768 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:02:14.769 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:02:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:02:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:02:15.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:02:15.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-mon[28085]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:15.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-mon[28085]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:02:15.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-mon[28085]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:02:15.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.905 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:15 smithi094 ceph-mon[31273]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:15.905 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:15 smithi094 ceph-mon[31273]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:02:15.905 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:15 smithi094 ceph-mon[31273]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:02:15.905 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.905 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:15.917 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0[42443]: 2024-04-26T21:02:15.663+0000 7f017ea00700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:02:15.917 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:02:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0[42443]: 2024-04-26T21:02:15.705+0000 7f0174800700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:02:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:15 smithi204 ceph-mon[32145]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:15 smithi204 ceph-mon[32145]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:02:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:15 smithi204 ceph-mon[32145]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:02:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:16 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:02:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:16 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:02:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:16 smithi094 ceph-mon[31273]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' 2024-04-26T21:02:16.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:16 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:02:16.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:16 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:02:16.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:16 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:16.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:16 smithi027 ceph-mon[28085]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' 2024-04-26T21:02:17.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:16 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:02:17.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:16 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:02:17.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:16 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:17.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:16 smithi204 ceph-mon[32145]: from='osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701]' entity='osd.0' 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357311 s, 293 MB/s 2024-04-26T21:02:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T21:02:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:17 smithi027 ceph-mon[28085]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:17 smithi027 ceph-mon[28085]: osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] boot 2024-04-26T21:02:17.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:17 smithi027 ceph-mon[28085]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:02:17.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:17 smithi204 ceph-mon[32145]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:17 smithi204 ceph-mon[32145]: osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] boot 2024-04-26T21:02:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:17 smithi204 ceph-mon[32145]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:02:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:18.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:17 smithi094 ceph-mon[31273]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:02:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:17 smithi094 ceph-mon[31273]: osd.0 [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] boot 2024-04-26T21:02:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:17 smithi094 ceph-mon[31273]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:02:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:02:18.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-26T21:02:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:02:18.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:18 smithi027 ceph-mon[28085]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:02:18.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:18 smithi027 ceph-mon[28085]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:18 smithi204 ceph-mon[32145]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:02:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:18 smithi204 ceph-mon[32145]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:18 smithi094 ceph-mon[31273]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:02:19.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:18 smithi094 ceph-mon[31273]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi027 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:20.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi027 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi027 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 17089M 2024-04-26T21:02:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:20.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:20.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 -- 172.21.15.94:0/2575242266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1056c0 msgr2=0x7f6c3c105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 --2- 172.21.15.94:0/2575242266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c105ae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c300099c0 tx=0x7f6c30009cd0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 -- 172.21.15.94:0/2575242266 shutdown_connections 2024-04-26T21:02:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 --2- 172.21.15.94:0/2575242266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c107260 0x7f6c3c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 --2- 172.21.15.94:0/2575242266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c105ae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 --2- 172.21.15.94:0/2575242266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 -- 172.21.15.94:0/2575242266 >> 172.21.15.94:0/2575242266 conn(0x7f6c3c100c80 msgr2=0x7f6c3c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:02:20.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 -- 172.21.15.94:0/2575242266 shutdown_connections 2024-04-26T21:02:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.914+0000 7f6c44400700 1 -- 172.21.15.94:0/2575242266 wait complete. 2024-04-26T21:02:20.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 Processor -- start 2024-04-26T21:02:20.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 -- start start 2024-04-26T21:02:20.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:20.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 0x7f6c3c1a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c111010 con 0x7f6c3c1056c0 2024-04-26T21:02:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c110e90 con 0x7f6c3c1068c0 2024-04-26T21:02:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.915+0000 7f6c44400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c111190 con 0x7f6c3c107260 2024-04-26T21:02:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48334/0 (socket says 172.21.15.94:48334) 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 -- 172.21.15.94:0/4003728781 learned_addr learned my addr 172.21.15.94:0/4003728781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41a00700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c42400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 0x7f6c3c1a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 -- 172.21.15.94:0/4003728781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 msgr2=0x7f6c3c1a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 0x7f6c3c1a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 -- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 msgr2=0x7f6c3c19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c19b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41000700 1 -- 172.21.15.94:0/4003728781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1a3370 con 0x7f6c3c1068c0 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c42400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 0x7f6c3c1a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.916+0000 7f6c41a00700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:02:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c41000700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e9c0 tx=0x7f6c2800ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:02:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800cbd0 con 0x7f6c3c1068c0 2024-04-26T21:02:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c2800cd70 con 0x7f6c3c1068c0 2024-04-26T21:02:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1a3660 con 0x7f6c3c1068c0 2024-04-26T21:02:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28005680 con 0x7f6c3c1068c0 2024-04-26T21:02:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.917+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1a3ba0 con 0x7f6c3c1068c0 2024-04-26T21:02:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.918+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c06a110 con 0x7f6c3c1068c0 2024-04-26T21:02:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.920+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6c28004750 con 0x7f6c3c1068c0 2024-04-26T21:02:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.922+0000 7f6c3a000700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 0x7f6c240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.922+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f6c280cef60 con 0x7f6c3c1068c0 2024-04-26T21:02:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.922+0000 7f6c41a00700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 0x7f6c240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.923+0000 7f6c41a00700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 0x7f6c240a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c30006010 tx=0x7f6c30005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:02:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:20.925+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6c2809d440 con 0x7f6c3c1068c0 2024-04-26T21:02:21.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:21.164+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6c3c063860 con 0x7f6c240a3c30 2024-04-26T21:02:21.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:21 smithi027 ceph-mon[28085]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:21.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:21 smithi204 ceph-mon[32145]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:21.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:21 smithi094 ceph-mon[31273]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:22.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:22 smithi094 ceph-mon[31273]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:22.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:22.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:22.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:22 smithi027 ceph-mon[28085]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:22 smithi204 ceph-mon[32145]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:23.481 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:23 smithi094 ceph-mon[31273]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:23.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:23 smithi027 ceph-mon[28085]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:23.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:23 smithi204 ceph-mon[32145]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/2098424619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]': finished 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:02:25.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/2098424619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]': finished 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:02:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/2098424619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]: dispatch 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88f3050-7a38-4f9e-8719-47deef44eb6f"}]': finished 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:02:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:26.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:26 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/3272769684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:02:26.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:26 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/3272769684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:02:26.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:26 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/3272769684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:02:27.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:27 smithi027 ceph-mon[28085]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:27 smithi204 ceph-mon[32145]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:27.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:27 smithi094 ceph-mon[31273]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:29.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:29 smithi027 ceph-mon[28085]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:29 smithi204 ceph-mon[32145]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:29 smithi094 ceph-mon[31273]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:31.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:31 smithi027 ceph-mon[28085]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:31.500 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:31 smithi094 ceph-mon[31273]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:31 smithi204 ceph-mon[32145]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:33.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:33 smithi027 ceph-mon[28085]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:33 smithi204 ceph-mon[32145]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:33.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:33 smithi094 ceph-mon[31273]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:34.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:02:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:34.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:02:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:34.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:02:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:35.343 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:35 smithi094 ceph-mon[31273]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:35.343 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:35 smithi094 ceph-mon[31273]: Deploying daemon osd.1 on smithi094 2024-04-26T21:02:35.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:35 smithi027 ceph-mon[28085]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:35.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:35 smithi027 ceph-mon[28085]: Deploying daemon osd.1 on smithi094 2024-04-26T21:02:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:35 smithi204 ceph-mon[32145]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:35 smithi204 ceph-mon[32145]: Deploying daemon osd.1 on smithi094 2024-04-26T21:02:37.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:37 smithi027 ceph-mon[28085]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:37.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:37 smithi204 ceph-mon[32145]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:37.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:37 smithi094 ceph-mon[31273]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:39.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:39 smithi094 ceph-mon[31273]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:39.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:39 smithi027 ceph-mon[28085]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:39.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:39 smithi204 ceph-mon[32145]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:41.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:41 smithi027 ceph-mon[28085]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:41.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:41 smithi204 ceph-mon[32145]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:41.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:41 smithi094 ceph-mon[31273]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:43.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:43 smithi094 ceph-mon[31273]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:43.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:43.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:43 smithi027 ceph-mon[28085]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:43.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:43 smithi204 ceph-mon[32145]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:45.334 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:45 smithi094 ceph-mon[31273]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:45.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:45 smithi027 ceph-mon[28085]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:45 smithi204 ceph-mon[32145]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:46 smithi027 ceph-mon[28085]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:46 smithi027 ceph-mon[28085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:46 smithi204 ceph-mon[32145]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:46 smithi204 ceph-mon[32145]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.755 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.755 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:46.755 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:46 smithi094 ceph-mon[31273]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.755 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:46 smithi094 ceph-mon[31273]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:02:46.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.886+0000 7f6c3a000700 1 -- 172.21.15.94:0/4003728781 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c3c063860 con 0x7f6c240a3c30 2024-04-26T21:02:46.888 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 1 on host 'smithi094' 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 msgr2=0x7f6c240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 0x7f6c240a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c30006010 tx=0x7f6c30005c20 comp rx=0 tx=0).stop 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 msgr2=0x7f6c3c19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e9c0 tx=0x7f6c2800ed80 comp rx=0 tx=0).stop 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 shutdown_connections 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6c240a3c30 0x7f6c240a60f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c3c107260 0x7f6c3c1a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c3c1056c0 0x7f6c3c19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 --2- 172.21.15.94:0/4003728781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c3c1068c0 0x7f6c3c19bca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 >> 172.21.15.94:0/4003728781 conn(0x7f6c3c100c80 msgr2=0x7f6c3c109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:02:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 shutdown_connections 2024-04-26T21:02:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:02:46.890+0000 7f6c44400700 1 -- 172.21.15.94:0/4003728781 wait complete. 2024-04-26T21:02:47.649 DEBUG:teuthology.orchestra.run.smithi094:osd.1> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:02:47.656 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-04-26T21:02:47.656 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:02:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:47.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:47.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:48.094 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:02:48.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:48 smithi094 ceph-mon[31273]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:02:48.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:48 smithi094 ceph-mon[31273]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:02:48.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:48.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:48.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:48 smithi027 ceph-mon[28085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:02:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:48 smithi027 ceph-mon[28085]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:02:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:48 smithi204 ceph-mon[32145]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:02:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:48 smithi204 ceph-mon[32145]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:02:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:49.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:02:49.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:02:49.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:49.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] boot 2024-04-26T21:02:49.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:02:49.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] boot 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:02:49.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: pgmap v85: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: osd.1 [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] boot 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:02:50.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:50 smithi094 ceph-mon[31273]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:02:50.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:50 smithi204 ceph-mon[32145]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0351562 s, 298 MB/s 2024-04-26T21:02:50.786 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-26T21:02:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:50 smithi027 ceph-mon[28085]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:02:51.518 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-04-26T21:02:51.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:51 smithi204 ceph-mon[32145]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:51.955 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:02:51.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:51 smithi027 ceph-mon[28085]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:52.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:51 smithi094 ceph-mon[31273]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:53.610 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi094 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:53.611 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi094 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi094 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:02:53.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-26T21:02:53.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:53.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:53.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:53.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:02:54.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 -- 172.21.15.204:0/154824930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 msgr2=0x7f4f240758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:54.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 --2- 172.21.15.204:0/154824930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f240758b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400b0a0 tx=0x7f4f1400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 -- 172.21.15.204:0/154824930 shutdown_connections 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 --2- 172.21.15.204:0/154824930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f2410d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 --2- 172.21.15.204:0/154824930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f240758b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 --2- 172.21.15.204:0/154824930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 0x7f4f24076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 -- 172.21.15.204:0/154824930 >> 172.21.15.204:0/154824930 conn(0x7f4f240fe950 msgr2=0x7f4f24100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 -- 172.21.15.204:0/154824930 shutdown_connections 2024-04-26T21:02:54.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.085+0000 7f4f29c00700 1 -- 172.21.15.204:0/154824930 wait complete. 2024-04-26T21:02:54.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 Processor -- start 2024-04-26T21:02:54.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 -- start start 2024-04-26T21:02:54.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 0x7f4f2419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:54.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:54.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f241a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24110f20 con 0x7f4f24075e80 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24110da0 con 0x7f4f2410b280 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.086+0000 7f4f29c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241110a0 con 0x7f4f24077450 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47572/0 (socket says 172.21.15.204:47572) 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 -- 172.21.15.204:0/2922442087 learned_addr learned my addr 172.21.15.204:0/2922442087 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f23e00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 0x7f4f2419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f29200700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f241a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 -- 172.21.15.204:0/2922442087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 msgr2=0x7f4f241a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f241a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 -- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 msgr2=0x7f4f2419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 0x7f4f2419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:02:54.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f1b400700 1 -- 172.21.15.204:0/2922442087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f241a3290 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.087+0000 7f4f29200700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f241a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f1b400700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d9b0 tx=0x7f4f0c00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c017030 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f0c0044b0 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0038e0 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f241a3580 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.088+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a3b10 con 0x7f4f24077450 2024-04-26T21:02:54.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.089+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2406a110 con 0x7f4f24077450 2024-04-26T21:02:54.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.093+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f4f0c01a070 con 0x7f4f24077450 2024-04-26T21:02:54.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.094+0000 7f4f21600700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 0x7f4f080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:02:54.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.094+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f4f0c0ce870 con 0x7f4f24077450 2024-04-26T21:02:54.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.095+0000 7f4f23e00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 0x7f4f080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:02:54.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.096+0000 7f4f23e00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 0x7f4f080a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ed0 tx=0x7f4f140061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:02:54.099 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.098+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4f0c09cb20 con 0x7f4f24077450 2024-04-26T21:02:54.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:02:54.340+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4f24063860 con 0x7f4f080a3c30 2024-04-26T21:02:54.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:54 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:54 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:54 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:54 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:54 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:54 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:54.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:54 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:02:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:54 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:02:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:54 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:55.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:55 smithi204 ceph-mon[32145]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:55.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:55 smithi204 ceph-mon[32145]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:55.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:55 smithi027 ceph-mon[28085]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:55 smithi027 ceph-mon[28085]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:55.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:55 smithi094 ceph-mon[31273]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:55 smithi094 ceph-mon[31273]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:02:57.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:57 smithi027 ceph-mon[28085]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:57 smithi204 ceph-mon[32145]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:57.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:57 smithi094 ceph-mon[31273]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:58 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/3427810258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:58 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:58 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]': finished 2024-04-26T21:02:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:58 smithi204 ceph-mon[32145]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:02:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:58 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:02:58.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:58 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/3427810258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:58 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:58 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]': finished 2024-04-26T21:02:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:58 smithi094 ceph-mon[31273]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:02:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:58 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:02:58.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:58 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/3427810258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:58 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]: dispatch 2024-04-26T21:02:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:58 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "689b4746-40ea-4411-bdb5-2aeedd58b38e"}]': finished 2024-04-26T21:02:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:58 smithi027 ceph-mon[28085]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:02:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:02:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:59 smithi204 ceph-mon[32145]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:02:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/1986387034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:02:59.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:59 smithi094 ceph-mon[31273]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:59.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:02:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/1986387034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:02:59.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:59 smithi027 ceph-mon[28085]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:02:59.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:02:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/1986387034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:03:01.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:01 smithi204 ceph-mon[32145]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:01.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:01 smithi094 ceph-mon[31273]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:01.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:01 smithi027 ceph-mon[28085]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:03.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:03 smithi204 ceph-mon[32145]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:03.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:03 smithi094 ceph-mon[31273]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:03.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:03 smithi027 ceph-mon[28085]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:05.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:05 smithi094 ceph-mon[31273]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:05.820 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:05 smithi204 ceph-mon[32145]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:05.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:05 smithi027 ceph-mon[28085]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:07 smithi204 ceph-mon[32145]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:07 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:03:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:07 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:07.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:07 smithi094 ceph-mon[31273]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:07 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:03:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:07 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:07.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:07 smithi027 ceph-mon[28085]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:07 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:03:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:07 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:08.686 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:08 smithi204 ceph-mon[32145]: Deploying daemon osd.2 on smithi204 2024-04-26T21:03:08.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:08 smithi094 ceph-mon[31273]: Deploying daemon osd.2 on smithi204 2024-04-26T21:03:08.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:08 smithi027 ceph-mon[28085]: Deploying daemon osd.2 on smithi204 2024-04-26T21:03:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:09 smithi204 ceph-mon[32145]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:09.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:09 smithi027 ceph-mon[28085]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:10.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:09 smithi094 ceph-mon[31273]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:11.921 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:11 smithi204 ceph-mon[32145]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:11.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:11 smithi027 ceph-mon[28085]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:12.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:11 smithi094 ceph-mon[31273]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:13.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:13 smithi027 ceph-mon[28085]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:14.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:13 smithi204 ceph-mon[32145]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:14.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:13 smithi094 ceph-mon[31273]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:15.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:15 smithi027 ceph-mon[28085]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:15.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:15.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:15.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:16.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:15 smithi204 ceph-mon[32145]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:16.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:15 smithi094 ceph-mon[31273]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:16.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:16.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:16.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:17 smithi204 ceph-mon[32145]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:17 smithi027 ceph-mon[28085]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:18.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:17 smithi094 ceph-mon[31273]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:19.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:19 smithi027 ceph-mon[28085]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:19 smithi027 ceph-mon[28085]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:19 smithi027 ceph-mon[28085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:19 smithi204 ceph-mon[32145]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:19 smithi204 ceph-mon[32145]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:19 smithi204 ceph-mon[32145]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:19 smithi094 ceph-mon[31273]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:19 smithi094 ceph-mon[31273]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:19 smithi094 ceph-mon[31273]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:03:19.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.812+0000 7f4f21600700 1 -- 172.21.15.204:0/2922442087 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f24063860 con 0x7f4f080a3c30 2024-04-26T21:03:19.814 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-04-26T21:03:19.816 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 msgr2=0x7f4f080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:19.816 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 0x7f4f080a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ed0 tx=0x7f4f140061f0 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 msgr2=0x7f4f2419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d9b0 tx=0x7f4f0c00dd70 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 shutdown_connections 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4f080a3c30 0x7f4f080a60f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2410b280 0x7f4f241a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f24075e80 0x7f4f2419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 --2- 172.21.15.204:0/2922442087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f24077450 0x7f4f2419bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 >> 172.21.15.204:0/2922442087 conn(0x7f4f240fe950 msgr2=0x7f4f24107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 shutdown_connections 2024-04-26T21:03:19.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:03:19.816+0000 7f4f29c00700 1 -- 172.21.15.204:0/2922442087 wait complete. 2024-04-26T21:03:20.569 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:03:20.574 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-04-26T21:03:20.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:03:20.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:20.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:20.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:03:20.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:20.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:03:20.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:20.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:20.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:20.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:21.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:03:21.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:21 smithi204 ceph-mon[32145]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:21.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:21 smithi204 ceph-mon[32145]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:03:21.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:21 smithi204 ceph-mon[32145]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:03:21.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:21.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:21.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:21 smithi027 ceph-mon[28085]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:21.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:21 smithi027 ceph-mon[28085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:03:21.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:21 smithi027 ceph-mon[28085]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:03:21.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:21.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:21 smithi094 ceph-mon[31273]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:21 smithi094 ceph-mon[31273]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:03:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:21 smithi094 ceph-mon[31273]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:03:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:22.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:03:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:03:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] boot 2024-04-26T21:03:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:03:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:22 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] boot 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:22 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: osd.2 [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] boot 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:03:23.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:22 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:23.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:23 smithi027 ceph-mon[28085]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:23.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:23.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:23 smithi027 ceph-mon[28085]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:03:23.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:23 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:23.927 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:23 smithi094 ceph-mon[31273]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:23.928 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:23 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:23.928 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:23 smithi094 ceph-mon[31273]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:03:23.928 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:23 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:23 smithi204 ceph-mon[32145]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:03:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:23 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:23 smithi204 ceph-mon[32145]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:03:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:23 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46788]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46788]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46794]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46794]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.179 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46794]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.180 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:03:23 smithi027 sudo[46794]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0398533 s, 263 MB/s 2024-04-26T21:03:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T21:03:24.276 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39702]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39702]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39706]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39706]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39706]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.277 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39706]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41265]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41265]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41265]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41265]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41268]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46899]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46899]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46899]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46899]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46902]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 sudo[46902]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41275]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41275]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41278]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41278]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41278]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 sudo[41278]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:24 smithi204 ceph-mon[32145]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39709]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39709]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39709]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39709]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39715]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39715]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 sudo[39715]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.758 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.759 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:24 smithi094 ceph-mon[31273]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:03:24.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:24 smithi027 ceph-mon[28085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:03:25.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-04-26T21:03:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:03:25.880 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:25.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:25.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:25.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:25.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:25.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:25.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:26.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:26.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:26.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:03:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:03:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:26 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi204 2024-04-26T21:03:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:26 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 17089M 2024-04-26T21:03:27.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:26 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi204 2024-04-26T21:03:27.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:26 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 17089M 2024-04-26T21:03:27.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:26 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi204 2024-04-26T21:03:27.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:26 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 17089M 2024-04-26T21:03:27.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 -- 172.21.15.27:0/2104281581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103480 msgr2=0x7f51a41038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 --2- 172.21.15.27:0/2104281581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103480 0x7f51a41038a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51980099c0 tx=0x7f5198009cd0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 -- 172.21.15.27:0/2104281581 shutdown_connections 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 --2- 172.21.15.27:0/2104281581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 0x7f51a410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 --2- 172.21.15.27:0/2104281581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4103480 0x7f51a41038a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 --2- 172.21.15.27:0/2104281581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4104680 0x7f51a4104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 -- 172.21.15.27:0/2104281581 >> 172.21.15.27:0/2104281581 conn(0x7f51a40fea20 msgr2=0x7f51a4100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 -- 172.21.15.27:0/2104281581 shutdown_connections 2024-04-26T21:03:27.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.515+0000 7f51abc00700 1 -- 172.21.15.27:0/2104281581 wait complete. 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.516+0000 7f51abc00700 1 Processor -- start 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.516+0000 7f51abc00700 1 -- start start 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4103480 0x7f51a40771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 0x7f51a4075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410ee20 con 0x7f51a4104680 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410eca0 con 0x7f51a4103480 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51abc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410efa0 con 0x7f51a4105020 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56952/0 (socket says 172.21.15.27:56952) 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51a3e00700 1 -- 172.21.15.27:0/3809157308 learned_addr learned my addr 172.21.15.27:0/3809157308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51a9200700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4103480 0x7f51a40771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:03:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.517+0000 7f51a9c00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 0x7f51a4075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 -- 172.21.15.27:0/3809157308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 msgr2=0x7f51a4075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 0x7f51a4075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 -- 172.21.15.27:0/3809157308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4103480 msgr2=0x7f51a40771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4103480 0x7f51a40771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 -- 172.21.15.27:0/3809157308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a4076600 con 0x7f51a4104680 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a3e00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f519400d950 tx=0x7f519400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194016070 con 0x7f51a4104680 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5194010460 con 0x7f51a4104680 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a36c0 con 0x7f51a4104680 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519400f620 con 0x7f51a4104680 2024-04-26T21:03:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.518+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a3c00 con 0x7f51a4104680 2024-04-26T21:03:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.520+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a406a110 con 0x7f51a4104680 2024-04-26T21:03:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.522+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5194019070 con 0x7f51a4104680 2024-04-26T21:03:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.523+0000 7f51a1600700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 0x7f518c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:03:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.523+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f51940cec40 con 0x7f51a4104680 2024-04-26T21:03:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.524+0000 7f51a9200700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 0x7f518c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:03:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.524+0000 7f51a9200700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 0x7f518c0a6180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5198009990 tx=0x7f519801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:03:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.529+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f519409c950 con 0x7f51a4104680 2024-04-26T21:03:27.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:27.771+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f51a4063860 con 0x7f518c0a3cc0 2024-04-26T21:03:27.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:27 smithi027 ceph-mon[28085]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:28.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:27 smithi204 ceph-mon[32145]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:28.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:27 smithi094 ceph-mon[31273]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:28.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:03:28.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:03:28.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:03:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:03:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:29.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:03:29.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:03:29.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:29.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:29 smithi027 ceph-mon[28085]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:03:29.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:29 smithi027 ceph-mon[28085]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:29 smithi204 ceph-mon[32145]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:03:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:29 smithi204 ceph-mon[32145]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:30.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:29 smithi094 ceph-mon[31273]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:03:30.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:29 smithi094 ceph-mon[31273]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:30.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:30 smithi027 ceph-mon[28085]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-04-26T21:03:31.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:30 smithi204 ceph-mon[32145]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-04-26T21:03:31.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:30 smithi094 ceph-mon[31273]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-04-26T21:03:33.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-04-26T21:03:33.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]: dispatch 2024-04-26T21:03:33.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]': finished 2024-04-26T21:03:33.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:03:33.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:33.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:32 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1603865306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]: dispatch 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]': finished 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:32 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1603865306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]: dispatch 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2074273676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496145b1-6477-4d0e-a222-c6222e95eac0"}]': finished 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:33.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:32 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1603865306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:03:35.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:34 smithi204 ceph-mon[32145]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-04-26T21:03:35.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:34 smithi094 ceph-mon[31273]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-04-26T21:03:35.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:34 smithi027 ceph-mon[28085]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-04-26T21:03:37.223 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:37 smithi027 ceph-mon[28085]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:37.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:37 smithi094 ceph-mon[31273]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:37.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:37 smithi204 ceph-mon[32145]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:39.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:39 smithi027 ceph-mon[28085]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:39.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:39 smithi094 ceph-mon[31273]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:39.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:39 smithi204 ceph-mon[32145]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-26T21:03:41.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:41 smithi027 ceph-mon[28085]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:41 smithi094 ceph-mon[31273]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:41.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:41 smithi204 ceph-mon[32145]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:42.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:03:42.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:42.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:03:42.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:42.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:03:42.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:43.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:43 smithi027 ceph-mon[28085]: Deploying daemon osd.3 on smithi027 2024-04-26T21:03:43.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:43 smithi027 ceph-mon[28085]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:43.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:43 smithi204 ceph-mon[32145]: Deploying daemon osd.3 on smithi027 2024-04-26T21:03:43.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:43 smithi204 ceph-mon[32145]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:43.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:43 smithi094 ceph-mon[31273]: Deploying daemon osd.3 on smithi027 2024-04-26T21:03:43.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:43 smithi094 ceph-mon[31273]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:45.203 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:45 smithi027 ceph-mon[28085]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:45.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:45 smithi204 ceph-mon[32145]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:45.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:45 smithi094 ceph-mon[31273]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:47.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:47 smithi027 ceph-mon[28085]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:47 smithi204 ceph-mon[32145]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:47.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:47 smithi094 ceph-mon[31273]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:49.478 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:49 smithi027 ceph-mon[28085]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:49.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:49 smithi204 ceph-mon[32145]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:49.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:49 smithi094 ceph-mon[31273]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:51.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:51 smithi027 ceph-mon[28085]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:51 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:51 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:51 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:51 smithi204 ceph-mon[32145]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:51.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:51 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:51 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:51 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:51.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:51 smithi094 ceph-mon[31273]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:51.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:51 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:51 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:51.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:51 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:53.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:53 smithi204 ceph-mon[32145]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:53.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:53 smithi094 ceph-mon[31273]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:53.340 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:53 smithi027 ceph-mon[28085]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:55.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:55 smithi027 ceph-mon[28085]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:55.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.697 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:55 smithi027 ceph-mon[28085]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:03:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:55 smithi204 ceph-mon[32145]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:55 smithi204 ceph-mon[32145]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:03:55.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:55 smithi094 ceph-mon[31273]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:55 smithi094 ceph-mon[31273]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:03:56.234 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-04-26T21:03:56.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.231+0000 7f51a1600700 1 -- 172.21.15.27:0/3809157308 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51a4063860 con 0x7f518c0a3cc0 2024-04-26T21:03:56.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 msgr2=0x7f518c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:56.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 0x7f518c0a6180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5198009990 tx=0x7f519801a040 comp rx=0 tx=0).stop 2024-04-26T21:03:56.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 msgr2=0x7f51a4075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f519400d950 tx=0x7f519400dd10 comp rx=0 tx=0).stop 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 shutdown_connections 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f518c0a3cc0 0x7f518c0a6180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51a4105020 0x7f51a4075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51a4103480 0x7f51a40771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 --2- 172.21.15.27:0/3809157308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4104680 0x7f51a4075810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.235+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 >> 172.21.15.27:0/3809157308 conn(0x7f51a40fea20 msgr2=0x7f51a4107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.236+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 shutdown_connections 2024-04-26T21:03:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:03:56.236+0000 7f51abc00700 1 -- 172.21.15.27:0/3809157308 wait complete. 2024-04-26T21:03:56.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:03:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:03:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:03:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:03:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:03:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:03:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:03:56.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:57.116 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:03:57.121 INFO:tasks.cephadm:Deploying osd.4 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-26T21:03:57.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:03:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:03:57.603 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:57.603 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:03:57.603 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:03:57.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:03:57.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:03:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:57 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:03:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:57 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:57.986 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:03:57.607+0000 7fb636600700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:03:57.986 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:03:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:03:57.634+0000 7fb62c400700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:03:58.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:58 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:03:58.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:58 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:03:58.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:58.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:58 smithi027 ceph-mon[28085]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' 2024-04-26T21:03:58.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:58 smithi027 ceph-mon[28085]: osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] boot 2024-04-26T21:03:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:58 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:03:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:58 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:03:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:58 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:58 smithi204 ceph-mon[32145]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' 2024-04-26T21:03:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:58 smithi204 ceph-mon[32145]: osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] boot 2024-04-26T21:03:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:58 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:03:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:58 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:03:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:58 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:58 smithi094 ceph-mon[31273]: from='osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482]' entity='osd.3' 2024-04-26T21:03:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:58 smithi094 ceph-mon[31273]: osd.3 [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] boot 2024-04-26T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:59 smithi027 ceph-mon[28085]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:59 smithi027 ceph-mon[28085]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:03:59 smithi027 ceph-mon[28085]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:03:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:59 smithi204 ceph-mon[32145]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:59 smithi204 ceph-mon[32145]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:03:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:03:59 smithi204 ceph-mon[32145]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:03:59.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:59 smithi094 ceph-mon[31273]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:03:59.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:59 smithi094 ceph-mon[31273]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:03:59.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:03:59.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:03:59 smithi094 ceph-mon[31273]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:04:00.359 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:04:00.360 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:04:00.360 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T21:04:00.360 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T21:04:00.360 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359936 s, 291 MB/s 2024-04-26T21:04:00.360 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T21:04:01.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-26T21:04:01.426 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:04:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:01 smithi027 ceph-mon[28085]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:01 smithi027 ceph-mon[28085]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:04:01.751 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:01 smithi094 ceph-mon[31273]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:01.751 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:01 smithi094 ceph-mon[31273]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:04:01.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:01 smithi204 ceph-mon[32145]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:01.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:01 smithi204 ceph-mon[32145]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:04:02.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:04:02.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:02.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:02.674 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.674 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:02.674 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:02.674 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:02.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:03.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:03.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 -- 172.21.15.94:0/3081270311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 msgr2=0x7fa1a8106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 --2- 172.21.15.94:0/3081270311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a8106330 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa198009990 tx=0x7fa198009ca0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 -- 172.21.15.94:0/3081270311 shutdown_connections 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 --2- 172.21.15.94:0/3081270311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a81091a0 0x7fa1a810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 --2- 172.21.15.94:0/3081270311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a8106330 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 --2- 172.21.15.94:0/3081270311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a8106870 0x7fa1a8108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 -- 172.21.15.94:0/3081270311 >> 172.21.15.94:0/3081270311 conn(0x7fa1a80fd970 msgr2=0x7fa1a80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 -- 172.21.15.94:0/3081270311 shutdown_connections 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.455+0000 7fa1ad200700 1 -- 172.21.15.94:0/3081270311 wait complete. 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.456+0000 7fa1ad200700 1 Processor -- start 2024-04-26T21:04:03.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.456+0000 7fa1ad200700 1 -- start start 2024-04-26T21:04:03.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:03.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 0x7fa1a8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 0x7fa1a8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810ee70 con 0x7fa1a8106870 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810ecf0 con 0x7fa1a8103f40 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1ad200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810eff0 con 0x7fa1a81091a0 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55970/0 (socket says 172.21.15.94:55970) 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a5e00700 1 -- 172.21.15.94:0/194828196 learned_addr learned my addr 172.21.15.94:0/194828196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a5400700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 0x7fa1a8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a5e00700 1 -- 172.21.15.94:0/194828196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 msgr2=0x7fa1a8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.457+0000 7fa1a6800700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 0x7fa1a8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 0x7fa1a8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5e00700 1 -- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 msgr2=0x7fa1a8101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 0x7fa1a8101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5e00700 1 -- 172.21.15.94:0/194828196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8102ce0 con 0x7fa1a8103f40 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a6800700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 0x7fa1a8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5400700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 0x7fa1a8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1a5e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa198005910 tx=0x7fa198005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198027070 con 0x7fa1a8103f40 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa198004060 con 0x7fa1a8103f40 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa1ad200700 1 -- 172.21.15.94:0/194828196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a81a35f0 con 0x7fa1a8103f40 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.458+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198022d20 con 0x7fa1a8103f40 2024-04-26T21:04:03.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.459+0000 7fa1ad200700 1 -- 172.21.15.94:0/194828196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a3a40 con 0x7fa1a8103f40 2024-04-26T21:04:03.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.460+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa180005330 con 0x7fa1a8103f40 2024-04-26T21:04:03.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.462+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa19802a070 con 0x7fa1a8103f40 2024-04-26T21:04:03.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.463+0000 7fa19ea00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 0x7fa18c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:03.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.463+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7fa1980d0970 con 0x7fa1a8103f40 2024-04-26T21:04:03.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.464+0000 7fa1a5400700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 0x7fa18c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:03.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.465+0000 7fa1a5400700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 0x7fa18c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa194005fd0 tx=0x7fa194005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:04:03.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.466+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa198031020 con 0x7fa1a8103f40 2024-04-26T21:04:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:03.701+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa180000bc0 con 0x7fa18c0a3c40 2024-04-26T21:04:03.703 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:03 smithi094 ceph-mon[31273]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:03.703 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:03 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi027 2024-04-26T21:04:03.703 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:03 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T21:04:03.703 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:03 smithi094 ceph-mon[31273]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:04:03.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:03 smithi027 ceph-mon[28085]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:03.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:03 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi027 2024-04-26T21:04:03.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:03 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T21:04:03.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:03 smithi027 ceph-mon[28085]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:04:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:03 smithi204 ceph-mon[32145]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:04.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:03 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi027 2024-04-26T21:04:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:03 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 8544M 2024-04-26T21:04:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:03 smithi204 ceph-mon[32145]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:04:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:04.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:04.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:04.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:05 smithi094 ceph-mon[31273]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:05 smithi094 ceph-mon[31273]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:05.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:05 smithi027 ceph-mon[28085]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:05.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:05 smithi027 ceph-mon[28085]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:06.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:05 smithi204 ceph-mon[32145]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:05 smithi204 ceph-mon[32145]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:07.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:07 smithi027 ceph-mon[28085]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:04:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:07 smithi204 ceph-mon[32145]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:04:08.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:07 smithi094 ceph-mon[31273]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/855868867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]': finished 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:08.810 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:08 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/845880490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/855868867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]': finished 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:09.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:08 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/845880490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/855868867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]: dispatch 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62db123-2dc9-4592-ad16-f2fe0723e9cd"}]': finished 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:09.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:08 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/845880490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:09 smithi027 ceph-mon[28085]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:04:10.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:09 smithi204 ceph-mon[32145]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:04:10.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:09 smithi094 ceph-mon[31273]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:04:11.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:11 smithi027 ceph-mon[28085]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:04:12.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:11 smithi204 ceph-mon[32145]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:04:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:11 smithi094 ceph-mon[31273]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:04:13.861 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:13 smithi094 ceph-mon[31273]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:04:13.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:13 smithi027 ceph-mon[28085]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:04:14.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:13 smithi204 ceph-mon[32145]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:04:14.943 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:14 smithi094 ceph-mon[31273]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:04:14.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:14 smithi027 ceph-mon[28085]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:04:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:14 smithi204 ceph-mon[32145]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:04:17.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:17 smithi094 ceph-mon[31273]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:17.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:04:17.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:17.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:17 smithi094 ceph-mon[31273]: Deploying daemon osd.4 on smithi094 2024-04-26T21:04:17.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:17 smithi027 ceph-mon[28085]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:17.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:04:17.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:17.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:17 smithi027 ceph-mon[28085]: Deploying daemon osd.4 on smithi094 2024-04-26T21:04:17.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:17 smithi204 ceph-mon[32145]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:17.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:04:17.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:17.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:17 smithi204 ceph-mon[32145]: Deploying daemon osd.4 on smithi094 2024-04-26T21:04:19.182 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:19 smithi094 ceph-mon[31273]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:19.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:19 smithi027 ceph-mon[28085]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:19.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:19 smithi204 ceph-mon[32145]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:21.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:21 smithi094 ceph-mon[31273]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:21.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:21 smithi027 ceph-mon[28085]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:21.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:21 smithi204 ceph-mon[32145]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:23.178 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:23 smithi094 ceph-mon[31273]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:23.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:23 smithi027 ceph-mon[28085]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:23.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:23 smithi204 ceph-mon[32145]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:25.633 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:25 smithi094 ceph-mon[31273]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:25.633 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:25.633 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:25.633 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:25.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:25 smithi027 ceph-mon[28085]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:25 smithi204 ceph-mon[32145]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:27.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:27 smithi094 ceph-mon[31273]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:27.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:27 smithi027 ceph-mon[28085]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:27 smithi204 ceph-mon[32145]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:28.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:28 smithi027 ceph-mon[28085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:28.986 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:28.987 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:28.987 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:28.987 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:28.987 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:28 smithi094 ceph-mon[31273]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:29.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:29.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:28 smithi204 ceph-mon[32145]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:04:29.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.067+0000 7fa19ea00700 1 -- 172.21.15.94:0/194828196 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa180000bc0 con 0x7fa18c0a3c40 2024-04-26T21:04:29.069 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 4 on host 'smithi094' 2024-04-26T21:04:29.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 msgr2=0x7fa18c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:29.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 0x7fa18c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa194005fd0 tx=0x7fa194005ee0 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 msgr2=0x7fa1a81038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa198005910 tx=0x7fa198005e80 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 shutdown_connections 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa18c0a3c40 0x7fa18c0a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a81091a0 0x7fa1a8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a8103f40 0x7fa1a81038a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 --2- 172.21.15.94:0/194828196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a8106870 0x7fa1a8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.071+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 >> 172.21.15.94:0/194828196 conn(0x7fa1a80fd970 msgr2=0x7fa1a8104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.072+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 shutdown_connections 2024-04-26T21:04:29.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:04:29.072+0000 7fa193e00700 1 -- 172.21.15.94:0/194828196 wait complete. 2024-04-26T21:04:29.891 DEBUG:teuthology.orchestra.run.smithi094:osd.4> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:04:29.895 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-04-26T21:04:29.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:04:30.140 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:04:30.140 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:30.141 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:30.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:04:30.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:30.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:30.336 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:04:31.153 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:30 smithi094 ceph-mon[31273]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:04:31.153 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:30 smithi094 ceph-mon[31273]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:04:31.153 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:30 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.153 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:30 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.154 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:30 smithi094 ceph-mon[31273]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:31.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:30 smithi204 ceph-mon[32145]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:04:31.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:30 smithi204 ceph-mon[32145]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:04:31.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:30 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:30 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.216 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:30 smithi204 ceph-mon[32145]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:31.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:30 smithi027 ceph-mon[28085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:30 smithi027 ceph-mon[28085]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:30 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:30 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:30 smithi027 ceph-mon[28085]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:04:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:31 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:04:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:31 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:04:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:31 smithi094 ceph-mon[31273]: osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] boot 2024-04-26T21:04:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:31 smithi094 ceph-mon[31273]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:04:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:31 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:32.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:31 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:04:32.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:31 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:04:32.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:31 smithi204 ceph-mon[32145]: osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] boot 2024-04-26T21:04:32.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:31 smithi204 ceph-mon[32145]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:04:32.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:31 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:32.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:31 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:04:32.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:31 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:04:32.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:31 smithi027 ceph-mon[28085]: osd.4 [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] boot 2024-04-26T21:04:32.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:31 smithi027 ceph-mon[28085]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:04:32.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:31 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0362502 s, 289 MB/s 2024-04-26T21:04:33.131 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-26T21:04:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:32 smithi204 ceph-mon[32145]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:04:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:32 smithi204 ceph-mon[32145]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:33.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:32 smithi027 ceph-mon[28085]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:04:33.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:32 smithi027 ceph-mon[28085]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:32 smithi094 ceph-mon[31273]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:04:33.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:32 smithi094 ceph-mon[31273]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:33.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-04-26T21:04:34.323 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:04:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi094 2024-04-26T21:04:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:35.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi094 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T21:04:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:35.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:35.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi094 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:35.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:35.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:36.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 -- 172.21.15.204:0/2116937113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 msgr2=0x7f72c8106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 --2- 172.21.15.204:0/2116937113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c8106290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009990 tx=0x7f72b8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 -- 172.21.15.204:0/2116937113 shutdown_connections 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 --2- 172.21.15.204:0/2116937113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c810b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 --2- 172.21.15.204:0/2116937113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c8106290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 --2- 172.21.15.204:0/2116937113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 -- 172.21.15.204:0/2116937113 >> 172.21.15.204:0/2116937113 conn(0x7f72c80fd830 msgr2=0x7f72c80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 -- 172.21.15.204:0/2116937113 shutdown_connections 2024-04-26T21:04:36.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.340+0000 7f72cd600700 1 -- 172.21.15.204:0/2116937113 wait complete. 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.341+0000 7f72cd600700 1 Processor -- start 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 -- start start 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c81a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f72c810ed70 con 0x7f72c8109100 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72c810ebf0 con 0x7f72c81067d0 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72cd600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f72c810eef0 con 0x7f72c8103ea0 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72c7e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72c7e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:37368/0 (socket says 172.21.15.204:37368) 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.342+0000 7f72c7e00700 1 -- 172.21.15.204:0/3772084992 learned_addr learned my addr 172.21.15.204:0/3772084992 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7400700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:36.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72ccc00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c81a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7e00700 1 -- 172.21.15.204:0/3772084992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 msgr2=0x7f72c819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7e00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7e00700 1 -- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 msgr2=0x7f72c81a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7e00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c81a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7e00700 1 -- 172.21.15.204:0/3772084992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c81a31f0 con 0x7f72c8103ea0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72ccc00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c81a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.343+0000 7f72c7400700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72c7e00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f72b8005f50 tx=0x7f72b800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8026070 con 0x7f72c8103ea0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72b8003da0 con 0x7f72c8103ea0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c81a3480 con 0x7f72c8103ea0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8021410 con 0x7f72c8103ea0 2024-04-26T21:04:36.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.344+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a3930 con 0x7f72c8103ea0 2024-04-26T21:04:36.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.346+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c806a110 con 0x7f72c8103ea0 2024-04-26T21:04:36.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.349+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f72b8029070 con 0x7f72c8103ea0 2024-04-26T21:04:36.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.350+0000 7f72c4c00700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 0x7f72ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:04:36.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.350+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f72b80d0550 con 0x7f72c8103ea0 2024-04-26T21:04:36.351 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.351+0000 7f72c7400700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 0x7f72ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:04:36.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.352+0000 7f72c7400700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 0x7f72ac0a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011490 tx=0x7f72b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:04:36.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.354+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f72b8013070 con 0x7f72c8103ea0 2024-04-26T21:04:36.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:04:36.592+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f72c8063860 con 0x7f72ac0a3cc0 2024-04-26T21:04:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:37 smithi027 ceph-mon[28085]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:37 smithi027 ceph-mon[28085]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:37 smithi204 ceph-mon[32145]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:37 smithi204 ceph-mon[32145]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:37.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:37 smithi094 ceph-mon[31273]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:37 smithi094 ceph-mon[31273]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:04:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:04:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:04:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:39.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:39 smithi027 ceph-mon[28085]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:39 smithi204 ceph-mon[32145]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:39.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:39 smithi094 ceph-mon[31273]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/1472812818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]': finished 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:04:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:41 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/1365365345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/1472812818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]': finished 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:04:41.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:41 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/1365365345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/1472812818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]: dispatch 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380e7876-c897-462a-96ee-0f6ef1239fb5"}]': finished 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:04:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:41 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/1365365345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:04:43.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:43 smithi027 ceph-mon[28085]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:43.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:43 smithi204 ceph-mon[32145]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:43.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:43 smithi094 ceph-mon[31273]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:45.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:45 smithi204 ceph-mon[32145]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:45.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:45 smithi027 ceph-mon[28085]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:45.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:45 smithi094 ceph-mon[31273]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:47.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:47 smithi027 ceph-mon[28085]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:47 smithi204 ceph-mon[32145]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:47.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:47 smithi094 ceph-mon[31273]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:49.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:49 smithi204 ceph-mon[32145]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:49.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:49 smithi094 ceph-mon[31273]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:49.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:49 smithi027 ceph-mon[28085]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:50.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:50 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:04:50.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:50 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:50.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:50 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:04:50.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:50 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:50 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:04:50.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:50 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:51.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:51 smithi094 ceph-mon[31273]: Deploying daemon osd.5 on smithi204 2024-04-26T21:04:51.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:51 smithi094 ceph-mon[31273]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:51.926 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:51 smithi204 ceph-mon[32145]: Deploying daemon osd.5 on smithi204 2024-04-26T21:04:51.926 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:51 smithi204 ceph-mon[32145]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:51.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:51 smithi027 ceph-mon[28085]: Deploying daemon osd.5 on smithi204 2024-04-26T21:04:51.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:51 smithi027 ceph-mon[28085]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:53.728 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:53 smithi204 ceph-mon[32145]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:53.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:53 smithi094 ceph-mon[31273]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:53.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:53 smithi027 ceph-mon[28085]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:55.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:55 smithi204 ceph-mon[32145]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:55 smithi094 ceph-mon[31273]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:55.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:55 smithi027 ceph-mon[28085]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:57.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:57 smithi204 ceph-mon[32145]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:57 smithi094 ceph-mon[31273]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:57.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:57 smithi027 ceph-mon[28085]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:58 smithi027 ceph-mon[28085]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:58.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:04:58 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:59.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:58 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:59.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:58 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:59.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:58 smithi204 ceph-mon[32145]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:59.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:04:58 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:59.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:58 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:59.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:58 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:04:59.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:58 smithi094 ceph-mon[31273]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:04:59.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:04:58 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:01.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:01 smithi094 ceph-mon[31273]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:01.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:01 smithi027 ceph-mon[28085]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:01.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:01 smithi204 ceph-mon[32145]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:02 smithi204 ceph-mon[32145]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:02 smithi027 ceph-mon[28085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:02 smithi094 ceph-mon[31273]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:05:02.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.617+0000 7f72c4c00700 1 -- 172.21.15.204:0/3772084992 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72c8063860 con 0x7f72ac0a3cc0 2024-04-26T21:05:02.619 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-04-26T21:05:02.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 msgr2=0x7f72ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:02.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 0x7f72ac0a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011490 tx=0x7f72b4009260 comp rx=0 tx=0).stop 2024-04-26T21:05:02.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 msgr2=0x7f72c819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:02.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f72b8005f50 tx=0x7f72b800bb20 comp rx=0 tx=0).stop 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 shutdown_connections 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f72ac0a3cc0 0x7f72ac0a6180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72c8109100 0x7f72c81a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72c8103ea0 0x7f72c819b590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 --2- 172.21.15.204:0/3772084992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c81067d0 0x7f72c819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 >> 172.21.15.204:0/3772084992 conn(0x7f72c80fd830 msgr2=0x7f72c81034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 shutdown_connections 2024-04-26T21:05:02.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:05:02.621+0000 7f72cd600700 1 -- 172.21.15.204:0/3772084992 wait complete. 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:03.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:05:03.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:03.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:03.596 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:05:03.601 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-04-26T21:05:03.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:05:04.079 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:05:04.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.377 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:05:04.093+0000 7f4eee600700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:05:04.378 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:05:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:05:04.127+0000 7f4ee4400700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:05:04.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:05:04.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:05:04.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:05:04.419 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:04 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:05:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:04 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.306 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: from='osd.5 ' entity='osd.5' 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] boot 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:05:05.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:05:05.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:05:05.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:05.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: from='osd.5 ' entity='osd.5' 2024-04-26T21:05:05.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] boot 2024-04-26T21:05:05.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:05:05.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: from='osd.5 ' entity='osd.5' 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: osd.5 [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] boot 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:05:05.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:05:06.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:06 smithi027 ceph-mon[28085]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:05:06.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:06 smithi094 ceph-mon[31273]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:05:06.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:06 smithi204 ceph-mon[32145]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339911 s, 308 MB/s 2024-04-26T21:05:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T21:05:07.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:07 smithi204 ceph-mon[32145]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:07.700 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:07 smithi204 ceph-mon[32145]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:05:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:07 smithi027 ceph-mon[28085]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:07 smithi027 ceph-mon[28085]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:05:07.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:07 smithi094 ceph-mon[31273]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:07 smithi094 ceph-mon[31273]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:05:08.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-04-26T21:05:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.702 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:08.703 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:08.703 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:08 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:08.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:08.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:08 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:08 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:09.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:09 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi204 2024-04-26T21:05:09.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:09 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 8544M 2024-04-26T21:05:09.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:09 smithi027 ceph-mon[28085]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:09.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:09 smithi027 ceph-mon[28085]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:05:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:09 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi204 2024-04-26T21:05:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:09 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 8544M 2024-04-26T21:05:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:09 smithi204 ceph-mon[32145]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:09 smithi204 ceph-mon[32145]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:05:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:09 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi204 2024-04-26T21:05:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:09 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 8544M 2024-04-26T21:05:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:09 smithi094 ceph-mon[31273]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:09 smithi094 ceph-mon[31273]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 -- 172.21.15.27:0/2818610437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010abb0 msgr2=0x7fa12010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 --2- 172.21.15.27:0/2818610437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010abb0 0x7fa12010b010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1100099c0 tx=0x7fa110009cd0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 -- 172.21.15.27:0/2818610437 shutdown_connections 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 --2- 172.21.15.27:0/2818610437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa12010b550 0x7fa120111e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 --2- 172.21.15.27:0/2818610437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1201099b0 0x7fa120109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 --2- 172.21.15.27:0/2818610437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010abb0 0x7fa12010b010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 -- 172.21.15.27:0/2818610437 >> 172.21.15.27:0/2818610437 conn(0x7fa120078980 msgr2=0x7fa12007adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:05:10.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.832+0000 7fa126000700 1 -- 172.21.15.27:0/2818610437 shutdown_connections 2024-04-26T21:05:10.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.833+0000 7fa126000700 1 -- 172.21.15.27:0/2818610437 wait complete. 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 Processor -- start 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 -- start start 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1201099b0 0x7fa12019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa12010abb0 0x7fa12019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201152d0 con 0x7fa12010b550 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa120115150 con 0x7fa1201099b0 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.834+0000 7fa126000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa120115450 con 0x7fa12010abb0 2024-04-26T21:05:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60158/0 (socket says 172.21.15.27:60158) 2024-04-26T21:05:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 -- 172.21.15.27:0/171819879 learned_addr learned my addr 172.21.15.27:0/171819879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:05:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa124c00700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1201099b0 0x7fa12019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa11fe00700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa12010abb0 0x7fa12019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 -- 172.21.15.27:0/171819879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa12010abb0 msgr2=0x7fa12019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa12010abb0 0x7fa12019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 -- 172.21.15.27:0/171819879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1201099b0 msgr2=0x7fa12019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1201099b0 0x7fa12019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.835+0000 7fa125600700 1 -- 172.21.15.27:0/171819879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1201a74e0 con 0x7fa12010b550 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa125600700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa118007da0 tx=0x7fa11800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118016020 con 0x7fa12010b550 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa11800ac50 con 0x7fa12010b550 2024-04-26T21:05:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180113c0 con 0x7fa12010b550 2024-04-26T21:05:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1201a77d0 con 0x7fa12010b550 2024-04-26T21:05:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.836+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201a7cc0 con 0x7fa12010b550 2024-04-26T21:05:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.837+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12006a110 con 0x7fa12010b550 2024-04-26T21:05:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.840+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa118019020 con 0x7fa12010b550 2024-04-26T21:05:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.841+0000 7fa11d600700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 0x7fa1080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.841+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7fa1180cf8d0 con 0x7fa12010b550 2024-04-26T21:05:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.842+0000 7fa124c00700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 0x7fa1080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:10.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.842+0000 7fa124c00700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 0x7fa1080a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa12010aa10 tx=0x7fa11400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:05:10.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:10.844+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa118020020 con 0x7fa12010b550 2024-04-26T21:05:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:11.087+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa120063860 con 0x7fa1080a3cc0 2024-04-26T21:05:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:11 smithi027 ceph-mon[28085]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T21:05:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:11 smithi204 ceph-mon[32145]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T21:05:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:11.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:11 smithi094 ceph-mon[31273]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T21:05:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:12.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:12 smithi027 ceph-mon[28085]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:12 smithi204 ceph-mon[32145]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:12.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:12 smithi094 ceph-mon[31273]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:13.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:13 smithi027 ceph-mon[28085]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:05:13.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:13 smithi204 ceph-mon[32145]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:05:13.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:13 smithi094 ceph-mon[31273]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:05:15.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:15 smithi027 ceph-mon[28085]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:05:15.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:15 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]: dispatch 2024-04-26T21:05:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:15 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]': finished 2024-04-26T21:05:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:15 smithi027 ceph-mon[28085]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:05:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:15 smithi204 ceph-mon[32145]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:05:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:15 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]: dispatch 2024-04-26T21:05:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:15 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]': finished 2024-04-26T21:05:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:15 smithi204 ceph-mon[32145]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:05:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:15.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:15 smithi094 ceph-mon[31273]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:05:15.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:15 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]: dispatch 2024-04-26T21:05:15.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:15 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1297270659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2c9acfd-f6a3-41e6-a3d3-b694781bd208"}]': finished 2024-04-26T21:05:15.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:15 smithi094 ceph-mon[31273]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:05:15.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:16.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:16 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/607831591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:16.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:16 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/607831591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:17.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:16 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/607831591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:17.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:17 smithi094 ceph-mon[31273]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:05:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:17 smithi027 ceph-mon[28085]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:05:18.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:17 smithi204 ceph-mon[32145]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:05:19.879 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:19 smithi027 ceph-mon[28085]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:05:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:19 smithi204 ceph-mon[32145]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:05:20.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:19 smithi094 ceph-mon[31273]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:05:21.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:21 smithi027 ceph-mon[28085]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:22.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:21 smithi204 ceph-mon[32145]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:21 smithi094 ceph-mon[31273]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:23.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:23 smithi027 ceph-mon[28085]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:23.828 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:23 smithi094 ceph-mon[31273]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:23 smithi204 ceph-mon[32145]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:25 smithi094 ceph-mon[31273]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:05:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:25.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:25 smithi027 ceph-mon[28085]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:25.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:05:25.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:25 smithi204 ceph-mon[32145]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:05:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:26.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:26 smithi027 ceph-mon[28085]: Deploying daemon osd.6 on smithi027 2024-04-26T21:05:27.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:26 smithi204 ceph-mon[32145]: Deploying daemon osd.6 on smithi027 2024-04-26T21:05:27.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:26 smithi094 ceph-mon[31273]: Deploying daemon osd.6 on smithi027 2024-04-26T21:05:27.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:27 smithi027 ceph-mon[28085]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:28.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:27 smithi204 ceph-mon[32145]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:28.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:27 smithi094 ceph-mon[31273]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:29.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:29 smithi027 ceph-mon[28085]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:29 smithi204 ceph-mon[32145]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:30.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:29 smithi094 ceph-mon[31273]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:31 smithi027 ceph-mon[28085]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:32.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:31 smithi204 ceph-mon[32145]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:32.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:31 smithi094 ceph-mon[31273]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:33.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:33 smithi027 ceph-mon[28085]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:34.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:33 smithi204 ceph-mon[32145]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:34.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:33 smithi094 ceph-mon[31273]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:35.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:35 smithi027 ceph-mon[28085]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:35.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:35.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:35 smithi094 ceph-mon[31273]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:35.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:35 smithi204 ceph-mon[32145]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:37.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:37 smithi027 ceph-mon[28085]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:37.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:37 smithi204 ceph-mon[32145]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:37.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:37 smithi094 ceph-mon[31273]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:38.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:38 smithi027 ceph-mon[28085]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:39.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:38 smithi204 ceph-mon[32145]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:39.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:38 smithi094 ceph-mon[31273]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:05:39.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.686+0000 7fa11d600700 1 -- 172.21.15.27:0/171819879 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa120063860 con 0x7fa1080a3cc0 2024-04-26T21:05:39.688 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.689+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 msgr2=0x7fa1080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.689+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 0x7fa1080a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa12010aa10 tx=0x7fa11400a3b0 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.689+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 msgr2=0x7fa1201a6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.689+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa118007da0 tx=0x7fa11800a340 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 shutdown_connections 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa1080a3cc0 0x7fa1080a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa12010b550 0x7fa1201a6e40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1201099b0 0x7fa12019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 --2- 172.21.15.27:0/171819879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa12010abb0 0x7fa12019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:39.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 >> 172.21.15.27:0/171819879 conn(0x7fa120078980 msgr2=0x7fa12010deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:05:39.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 shutdown_connections 2024-04-26T21:05:39.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:05:39.690+0000 7fa126000700 1 -- 172.21.15.27:0/171819879 wait complete. 2024-04-26T21:05:39.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:05:39.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:05:39.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:05:39.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:39.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:39.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:40.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:40.667 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:05:40.673 INFO:tasks.cephadm:Deploying osd.7 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-26T21:05:40.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:05:40.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-mon[28085]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:05:40.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-mon[28085]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:05:40.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:40.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:40.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-mon[28085]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:41.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:40 smithi094 ceph-mon[31273]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:05:41.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:40 smithi094 ceph-mon[31273]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:05:41.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:41.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:41.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:40 smithi094 ceph-mon[31273]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:41.104 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:05:41.148 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:05:40.875+0000 7f4cec400700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:05:41.148 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:05:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:05:40.909+0000 7f4ce2200700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:05:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:40 smithi204 ceph-mon[32145]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:05:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:40 smithi204 ceph-mon[32145]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:05:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:41.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:40 smithi204 ceph-mon[32145]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:42.001 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:41 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:05:42.001 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:41 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:05:42.001 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:42.001 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:41 smithi094 ceph-mon[31273]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' 2024-04-26T21:05:42.011 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:41 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:05:42.011 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:41 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:05:42.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:42.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:41 smithi027 ceph-mon[28085]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' 2024-04-26T21:05:42.190 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:41 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:05:42.190 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:41 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:05:42.190 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:42.191 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:41 smithi204 ceph-mon[32145]: from='osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972]' entity='osd.6' 2024-04-26T21:05:42.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:42.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:42 smithi204 ceph-mon[32145]: osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] boot 2024-04-26T21:05:42.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:42 smithi204 ceph-mon[32145]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:05:42.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:42.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:42 smithi204 ceph-mon[32145]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:43.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:43.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:42 smithi094 ceph-mon[31273]: osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] boot 2024-04-26T21:05:43.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:42 smithi094 ceph-mon[31273]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:05:43.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:43.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:42 smithi094 ceph-mon[31273]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:43.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:43.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:42 smithi027 ceph-mon[28085]: osd.6 [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] boot 2024-04-26T21:05:43.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:42 smithi027 ceph-mon[28085]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:05:43.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:05:43.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:42 smithi027 ceph-mon[28085]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0363582 s, 288 MB/s 2024-04-26T21:05:43.926 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T21:05:44.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:43 smithi027 ceph-mon[28085]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:05:44.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:43 smithi204 ceph-mon[32145]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:05:44.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:43 smithi094 ceph-mon[31273]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:05:44.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-26T21:05:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:05:45.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:44 smithi027 ceph-mon[28085]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:45.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:44 smithi204 ceph-mon[32145]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:45.267 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:44 smithi094 ceph-mon[31273]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:47.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 -- 172.21.15.94:0/1047742363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c075e80 msgr2=0x7fa52c076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:47.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 --2- 172.21.15.94:0/1047742363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c075e80 0x7fa52c076300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009990 tx=0x7fa51c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 -- 172.21.15.94:0/1047742363 shutdown_connections 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 --2- 172.21.15.94:0/1047742363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c10f9a0 0x7fa52c111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 --2- 172.21.15.94:0/1047742363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa52c077450 0x7fa52c0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 --2- 172.21.15.94:0/1047742363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c075e80 0x7fa52c076300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 -- 172.21.15.94:0/1047742363 >> 172.21.15.94:0/1047742363 conn(0x7fa52c0fe960 msgr2=0x7fa52c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 -- 172.21.15.94:0/1047742363 shutdown_connections 2024-04-26T21:05:47.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.192+0000 7fa533200700 1 -- 172.21.15.94:0/1047742363 wait complete. 2024-04-26T21:05:47.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.194+0000 7fa533200700 1 Processor -- start 2024-04-26T21:05:47.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.194+0000 7fa533200700 1 -- start start 2024-04-26T21:05:47.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.194+0000 7fa533200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 0x7fa52c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:47.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.194+0000 7fa533200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa533200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa52c10f9a0 0x7fa52c1a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa533200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c115370 con 0x7fa52c10f9a0 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa533200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1151f0 con 0x7fa52c077450 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa533200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1154f0 con 0x7fa52c075e80 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:47.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45194/0 (socket says 172.21.15.94:45194) 2024-04-26T21:05:47.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 -- 172.21.15.94:0/1410695916 learned_addr learned my addr 172.21.15.94:0/1410695916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T21:05:47.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531e00700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 0x7fa52c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:47.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 -- 172.21.15.94:0/1410695916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 msgr2=0x7fa52c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:47.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 0x7fa52c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 -- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa52c10f9a0 msgr2=0x7fa52c1a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa52c10f9a0 0x7fa52c1a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531400700 1 -- 172.21.15.94:0/1410695916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1a76e0 con 0x7fa52c077450 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.195+0000 7fa531e00700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 0x7fa52c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa531400700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa51c003820 tx=0x7fa51c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c027070 con 0x7fa52c077450 2024-04-26T21:05:47.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa51c004060 con 0x7fa52c077450 2024-04-26T21:05:47.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c1a7970 con 0x7fa52c077450 2024-04-26T21:05:47.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c022cb0 con 0x7fa52c077450 2024-04-26T21:05:47.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.196+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c1a7dc0 con 0x7fa52c077450 2024-04-26T21:05:47.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.197+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c06a110 con 0x7fa52c077450 2024-04-26T21:05:47.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.200+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa51c02a070 con 0x7fa52c077450 2024-04-26T21:05:47.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.202+0000 7fa526a00700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 0x7fa5140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:47.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.202+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fa51c0d0ff0 con 0x7fa52c077450 2024-04-26T21:05:47.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.202+0000 7fa531e00700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 0x7fa5140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:47.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.204+0000 7fa531e00700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 0x7fa5140a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa520005fd0 tx=0x7fa520005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:05:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.206+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa51c031020 con 0x7fa52c077450 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi027 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:47.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:47.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:05:47.445+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa52c063860 con 0x7fa5140a3c40 2024-04-26T21:05:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi027 2024-04-26T21:05:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:47.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:47.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi027 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 5696M 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:47.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:05:48.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:48 smithi094 ceph-mon[31273]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:48.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:48.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:48.416 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:48 smithi027 ceph-mon[28085]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:48.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:48 smithi204 ceph-mon[32145]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:05:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:05:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:05:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:49.425 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:49 smithi094 ceph-mon[31273]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:49.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:49 smithi027 ceph-mon[28085]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:49.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:49 smithi204 ceph-mon[32145]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:51.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:51 smithi027 ceph-mon[28085]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:51 smithi204 ceph-mon[32145]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:51.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:51 smithi094 ceph-mon[31273]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:52.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:52 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/641692185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:52 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:52 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]': finished 2024-04-26T21:05:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:52 smithi027 ceph-mon[28085]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:05:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:52 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:05:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:52 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/641692185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:52 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:52 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]': finished 2024-04-26T21:05:52.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:52 smithi204 ceph-mon[32145]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:05:52.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:52 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:05:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:52 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/641692185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:52 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]: dispatch 2024-04-26T21:05:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:52 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1776a60-ed45-4ece-8668-fe147a04409c"}]': finished 2024-04-26T21:05:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:52 smithi094 ceph-mon[31273]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:05:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:52 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:05:53.427 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:53 smithi094 ceph-mon[31273]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:53.427 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:53 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/9651871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:53.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:53 smithi027 ceph-mon[28085]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:53.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:53 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/9651871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:53.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:53 smithi204 ceph-mon[32145]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:53.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:53 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/9651871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:05:55.435 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:55 smithi094 ceph-mon[31273]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:55.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:55 smithi027 ceph-mon[28085]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:55.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:55 smithi204 ceph-mon[32145]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:57.400 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:57 smithi094 ceph-mon[31273]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:57.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:57 smithi027 ceph-mon[28085]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:57.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:57 smithi204 ceph-mon[32145]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:59.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:05:59 smithi094 ceph-mon[31273]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:59.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:05:59 smithi027 ceph-mon[28085]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:05:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:05:59 smithi204 ceph-mon[32145]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:01.396 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:01 smithi094 ceph-mon[31273]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:01.396 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:06:01.396 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:01.396 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:01 smithi094 ceph-mon[31273]: Deploying daemon osd.7 on smithi094 2024-04-26T21:06:01.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:01 smithi027 ceph-mon[28085]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:01.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:06:01.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:01.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:01 smithi027 ceph-mon[28085]: Deploying daemon osd.7 on smithi094 2024-04-26T21:06:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:01 smithi204 ceph-mon[32145]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:06:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:01 smithi204 ceph-mon[32145]: Deploying daemon osd.7 on smithi094 2024-04-26T21:06:03.458 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:03 smithi094 ceph-mon[31273]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:03.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:03 smithi027 ceph-mon[28085]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:03 smithi204 ceph-mon[32145]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:05.213 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:05 smithi094 ceph-mon[31273]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:05.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:05 smithi027 ceph-mon[28085]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:05 smithi204 ceph-mon[32145]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:07.447 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:07 smithi094 ceph-mon[31273]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:07.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:07 smithi027 ceph-mon[28085]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:07.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:07 smithi204 ceph-mon[32145]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:09.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:09 smithi027 ceph-mon[28085]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:09.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:09.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:09 smithi094 ceph-mon[31273]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:09.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:09 smithi204 ceph-mon[32145]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:11.488 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:11 smithi094 ceph-mon[31273]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:11.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:11 smithi204 ceph-mon[32145]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:11.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:11 smithi027 ceph-mon[28085]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.322+0000 7fa526a00700 1 -- 172.21.15.94:0/1410695916 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa52c063860 con 0x7fa5140a3c40 2024-04-26T21:06:13.324 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 7 on host 'smithi094' 2024-04-26T21:06:13.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:13 smithi094 ceph-mon[31273]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:13.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:13 smithi094 ceph-mon[31273]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:13.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:13 smithi094 ceph-mon[31273]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 msgr2=0x7fa5140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:13.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 0x7fa5140a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa520005fd0 tx=0x7fa520005ee0 comp rx=0 tx=0).stop 2024-04-26T21:06:13.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 msgr2=0x7fa52c1a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:13.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa51c003820 tx=0x7fa51c005e80 comp rx=0 tx=0).stop 2024-04-26T21:06:13.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 shutdown_connections 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5140a3c40 0x7fa5140a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa52c10f9a0 0x7fa52c1a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52c075e80 0x7fa52c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 --2- 172.21.15.94:0/1410695916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa52c077450 0x7fa52c1a0010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.327+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 >> 172.21.15.94:0/1410695916 conn(0x7fa52c0fe960 msgr2=0x7fa52c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.328+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 shutdown_connections 2024-04-26T21:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:06:13.328+0000 7fa533200700 1 -- 172.21.15.94:0/1410695916 wait complete. 2024-04-26T21:06:13.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:13 smithi027 ceph-mon[28085]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:13.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:13 smithi027 ceph-mon[28085]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:13.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:13 smithi027 ceph-mon[28085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:13 smithi204 ceph-mon[32145]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:13 smithi204 ceph-mon[32145]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:13.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:13 smithi204 ceph-mon[32145]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:06:14.122 DEBUG:teuthology.orchestra.run.smithi094:osd.7> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:06:14.127 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-04-26T21:06:14.127 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:06:14.326 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:14.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-mon[31273]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:14.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:14 smithi204 ceph-mon[32145]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:14.563 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:06:14.593 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:06:14.325+0000 7fb97d600700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:06:14.593 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:06:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:06:14.343+0000 7fb978400700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:14 smithi027 ceph-mon[28085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:06:15.484 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:15.485 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:15 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:15.724 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:15.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:15 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:15.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:06:15.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:06:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:06:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:06:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:15 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:16 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:16 smithi204 ceph-mon[32145]: osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] boot 2024-04-26T21:06:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:16 smithi204 ceph-mon[32145]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:06:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:16 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:16 smithi094 ceph-mon[31273]: osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] boot 2024-04-26T21:06:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:16 smithi094 ceph-mon[31273]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:06:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:16 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:16 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:16 smithi027 ceph-mon[28085]: osd.7 [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] boot 2024-04-26T21:06:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:16 smithi027 ceph-mon[28085]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:06:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:16 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:06:17.454 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:06:17.454 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:06:17.454 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-26T21:06:17.454 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-26T21:06:17.454 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354036 s, 296 MB/s 2024-04-26T21:06:17.455 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-26T21:06:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:17 smithi027 ceph-mon[28085]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:17 smithi027 ceph-mon[28085]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:06:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:17 smithi027 ceph-mon[28085]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:06:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:17 smithi204 ceph-mon[32145]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:17 smithi204 ceph-mon[32145]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:06:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:17 smithi204 ceph-mon[32145]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:06:17.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:17 smithi094 ceph-mon[31273]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:06:17.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:17 smithi094 ceph-mon[31273]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:06:17.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:17 smithi094 ceph-mon[31273]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:06:18.206 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-04-26T21:06:18.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:18 smithi204 ceph-mon[32145]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:06:18.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:18 smithi094 ceph-mon[31273]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:06:18.636 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:06:18.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:18 smithi027 ceph-mon[28085]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi094 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:19 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:06:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi094 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:19 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi094 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:19 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:20.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:20 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T21:06:20.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:20 smithi204 ceph-mon[32145]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.640+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2655042885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 msgr2=0x7ff9cc101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.640+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2655042885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc101430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009990 tx=0x7ff9c0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2655042885 shutdown_connections 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2655042885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9cc109090 0x7ff9cc10b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2655042885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc101430 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2655042885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 0x7ff9cc108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2655042885 >> 172.21.15.204:0/2655042885 conn(0x7ff9cc0fcba0 msgr2=0x7ff9cc0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:20.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2655042885 shutdown_connections 2024-04-26T21:06:20.643 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.641+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2655042885 wait complete. 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 Processor -- start 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 -- start start 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 0x7ff9cc102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9cc109090 0x7ff9cc103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:20.644 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10ecc0 con 0x7ff9cc109090 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10eb40 con 0x7ff9cc101a00 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d1e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10ee40 con 0x7ff9cc101010 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.642+0000 7ff9d0a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:58408/0 (socket says 172.21.15.204:58408) 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 -- 172.21.15.204:0/2685207134 learned_addr learned my addr 172.21.15.204:0/2685207134 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9cbe00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 0x7ff9cc102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 -- 172.21.15.204:0/2685207134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 msgr2=0x7ff9cc102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 0x7ff9cc102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 -- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9cc109090 msgr2=0x7ff9cc103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9cc109090 0x7ff9cc103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 -- 172.21.15.204:0/2685207134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc1038d0 con 0x7ff9cc101010 2024-04-26T21:06:20.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9d0a00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098e0 tx=0x7ff9c0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.643+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0027070 con 0x7ff9cc101010 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.644+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9c0004060 con 0x7ff9cc101010 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.644+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc1a34f0 con 0x7ff9cc101010 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.644+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0022d90 con 0x7ff9cc101010 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.644+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a3940 con 0x7ff9cc101010 2024-04-26T21:06:20.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.645+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc06a110 con 0x7ff9cc101010 2024-04-26T21:06:20.648 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.647+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9c002a070 con 0x7ff9cc101010 2024-04-26T21:06:20.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.648+0000 7ff9c9600700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 0x7ff9b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:20.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.649+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5397+0+0 (secure 0 0 0) 0x7ff9c00d10b0 con 0x7ff9cc101010 2024-04-26T21:06:20.650 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.649+0000 7ff9cbe00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 0x7ff9b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:20.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.650+0000 7ff9cbe00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 0x7ff9b40a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:20.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.651+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff9c0031020 con 0x7ff9cc101010 2024-04-26T21:06:20.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:20 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T21:06:20.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:20 smithi027 ceph-mon[28085]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:06:20.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:20 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-26T21:06:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:20 smithi094 ceph-mon[31273]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:06:20.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:20.886+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff9cc063860 con 0x7ff9b40a3c40 2024-04-26T21:06:21.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:21 smithi027 ceph-mon[28085]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:21.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:21 smithi204 ceph-mon[32145]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:21.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:21 smithi094 ceph-mon[31273]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:22 smithi204 ceph-mon[32145]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:22.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:22 smithi027 ceph-mon[28085]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:22 smithi094 ceph-mon[31273]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:23.677 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:23 smithi094 ceph-mon[31273]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:23.677 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:23 smithi094 ceph-mon[31273]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:06:23.677 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:23 smithi094 ceph-mon[31273]: Cluster is now healthy 2024-04-26T21:06:23.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:23 smithi027 ceph-mon[28085]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:23 smithi027 ceph-mon[28085]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:06:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:23 smithi027 ceph-mon[28085]: Cluster is now healthy 2024-04-26T21:06:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:23 smithi204 ceph-mon[32145]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:23 smithi204 ceph-mon[32145]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:06:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:23 smithi204 ceph-mon[32145]: Cluster is now healthy 2024-04-26T21:06:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/1724921507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]': finished 2024-04-26T21:06:25.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:06:25.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/1724921507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]': finished 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:06:25.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/1724921507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]: dispatch 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87f761-8ce7-4634-92e9-223f67e64af6"}]': finished 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:06:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:26.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:26 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/3110423523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:06:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:26 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/3110423523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:06:26.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:26 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/3110423523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:06:27.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:27 smithi027 ceph-mon[28085]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:27 smithi204 ceph-mon[32145]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:27.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:27 smithi094 ceph-mon[31273]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:29.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:29 smithi204 ceph-mon[32145]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:29.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:29 smithi027 ceph-mon[28085]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:29.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:29 smithi094 ceph-mon[31273]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:31.699 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:31 smithi204 ceph-mon[32145]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:31.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:31 smithi027 ceph-mon[28085]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:31.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:31 smithi094 ceph-mon[31273]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:33.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:33 smithi027 ceph-mon[28085]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:33.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:33 smithi204 ceph-mon[32145]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:33 smithi094 ceph-mon[31273]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:35.716 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:35 smithi204 ceph-mon[32145]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:35.716 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:06:35.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:35.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:35 smithi204 ceph-mon[32145]: Deploying daemon osd.8 on smithi204 2024-04-26T21:06:35.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:35 smithi027 ceph-mon[28085]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:35.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:06:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:35 smithi027 ceph-mon[28085]: Deploying daemon osd.8 on smithi204 2024-04-26T21:06:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:35 smithi094 ceph-mon[31273]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:06:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:35 smithi094 ceph-mon[31273]: Deploying daemon osd.8 on smithi204 2024-04-26T21:06:37.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:37 smithi204 ceph-mon[32145]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:37.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:37 smithi027 ceph-mon[28085]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:37.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:37 smithi094 ceph-mon[31273]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:39.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:39 smithi027 ceph-mon[28085]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:39 smithi204 ceph-mon[32145]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:39.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:39 smithi094 ceph-mon[31273]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:41.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:41 smithi027 ceph-mon[28085]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:41.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:41 smithi204 ceph-mon[32145]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:41.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:41 smithi094 ceph-mon[31273]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:43.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:43 smithi204 ceph-mon[32145]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:43.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:43.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:43.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:43 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:43.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:43 smithi027 ceph-mon[28085]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:43.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:43.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:43.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:43 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:44.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:43 smithi094 ceph-mon[31273]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:44.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:44.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:44.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:43 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:45.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:45 smithi027 ceph-mon[28085]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:46.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:45 smithi204 ceph-mon[32145]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:46.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:45 smithi094 ceph-mon[31273]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:47.202 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-04-26T21:06:47.202 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.200+0000 7ff9c9600700 1 -- 172.21.15.204:0/2685207134 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9cc063860 con 0x7ff9b40a3c40 2024-04-26T21:06:47.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 msgr2=0x7ff9b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 0x7ff9b40a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b80058e0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 msgr2=0x7ff9cc104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098e0 tx=0x7ff9c0005e80 comp rx=0 tx=0).stop 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 shutdown_connections 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff9b40a3c40 0x7ff9b40a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9cc109090 0x7ff9cc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9cc101010 0x7ff9cc104490 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 --2- 172.21.15.204:0/2685207134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9cc101a00 0x7ff9cc102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 >> 172.21.15.204:0/2685207134 conn(0x7ff9cc0fcba0 msgr2=0x7ff9cc104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.203+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 shutdown_connections 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:06:47.204+0000 7ff9d1e00700 1 -- 172.21.15.204:0/2685207134 wait complete. 2024-04-26T21:06:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:06:47.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:47.945 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:06:47.950 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-04-26T21:06:47.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:06:48.418 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:06:48.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:48.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:06:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:06:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:48 smithi027 ceph-mon[28085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:48.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:48.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:06:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:06:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:48 smithi204 ceph-mon[32145]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: from='osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:48 smithi094 ceph-mon[31273]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:06:49.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-mon[32145]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:49.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-mon[32145]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:06:49.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-mon[32145]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:06:49.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:49.929 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:49.929 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:06:49.664+0000 7f50aa600700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:06:49.929 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:06:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:06:49.715+0000 7f50a0400700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:06:49.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:49 smithi027 ceph-mon[28085]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:49 smithi027 ceph-mon[28085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:06:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:49 smithi027 ceph-mon[28085]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:06:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:49.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:49 smithi094 ceph-mon[31273]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:49 smithi094 ceph-mon[31273]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:06:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:49 smithi094 ceph-mon[31273]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:06:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:50 smithi027 ceph-mon[28085]: from='osd.8 ' entity='osd.8' 2024-04-26T21:06:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:06:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:06:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:06:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:51.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:51.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:50 smithi204 ceph-mon[32145]: from='osd.8 ' entity='osd.8' 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:50 smithi094 ceph-mon[31273]: from='osd.8 ' entity='osd.8' 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367496 s, 285 MB/s 2024-04-26T21:06:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-26T21:06:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:51 smithi027 ceph-mon[28085]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:51 smithi027 ceph-mon[28085]: osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] boot 2024-04-26T21:06:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:51 smithi027 ceph-mon[28085]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:06:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:51 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:51 smithi027 ceph-mon[28085]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:06:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:51 smithi204 ceph-mon[32145]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:51 smithi204 ceph-mon[32145]: osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] boot 2024-04-26T21:06:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:51 smithi204 ceph-mon[32145]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:06:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:51 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:51 smithi204 ceph-mon[32145]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:06:52.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:51 smithi094 ceph-mon[31273]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:06:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:51 smithi094 ceph-mon[31273]: osd.8 [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] boot 2024-04-26T21:06:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:51 smithi094 ceph-mon[31273]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:06:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:51 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:06:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:51 smithi094 ceph-mon[31273]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:06:52.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-04-26T21:06:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:06:53.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi204 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 5696M 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:53.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:53.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:53.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:53.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:53 smithi027 ceph-mon[28085]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:06:54.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi204 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 5696M 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:54.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.004 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:53 smithi204 ceph-mon[32145]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:06:54.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:54.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi204 2024-04-26T21:06:54.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 5696M 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:54.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:06:54.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:53 smithi094 ceph-mon[31273]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 -- 172.21.15.27:0/3832790018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc01090a0 msgr2=0x7fddc010b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 --2- 172.21.15.27:0/3832790018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc01090a0 0x7fddc010b510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009990 tx=0x7fddb8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 -- 172.21.15.27:0/3832790018 shutdown_connections 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 --2- 172.21.15.27:0/3832790018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc01090a0 0x7fddc010b510 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 --2- 172.21.15.27:0/3832790018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0103e40 0x7fddc0106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 --2- 172.21.15.27:0/3832790018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0106770 0x7fddc0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 -- 172.21.15.27:0/3832790018 >> 172.21.15.27:0/3832790018 conn(0x7fddc00fd830 msgr2=0x7fddc00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 -- 172.21.15.27:0/3832790018 shutdown_connections 2024-04-26T21:06:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.141+0000 7fddc5a00700 1 -- 172.21.15.27:0/3832790018 wait complete. 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.142+0000 7fddc5a00700 1 Processor -- start 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 -- start start 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0103e40 0x7fddc0197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01090a0 0x7fddc0194900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010ed70 con 0x7fddc0106770 2024-04-26T21:06:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010ebf0 con 0x7fddc01090a0 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddc5a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010eef0 con 0x7fddc0103e40 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddb7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddb7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48886/0 (socket says 172.21.15.27:48886) 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.143+0000 7fddb7400700 1 -- 172.21.15.27:0/2877046964 learned_addr learned my addr 172.21.15.27:0/2877046964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddbfe00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0103e40 0x7fddc0197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddc5000700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01090a0 0x7fddc0194900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 -- 172.21.15.27:0/2877046964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0103e40 msgr2=0x7fddc0197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0103e40 0x7fddc0197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 -- 172.21.15.27:0/2877046964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01090a0 msgr2=0x7fddc0194900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01090a0 0x7fddc0194900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 -- 172.21.15.27:0/2877046964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc01951d0 con 0x7fddc0106770 2024-04-26T21:06:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.144+0000 7fddb7400700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e9c0 tx=0x7fdda800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.145+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda800cbd0 con 0x7fddc0106770 2024-04-26T21:06:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.145+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdda800ee60 con 0x7fddc0106770 2024-04-26T21:06:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.145+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80054b0 con 0x7fddc0106770 2024-04-26T21:06:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.145+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc01a7a10 con 0x7fddc0106770 2024-04-26T21:06:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.145+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01a7e10 con 0x7fddc0106770 2024-04-26T21:06:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.146+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc006a110 con 0x7fddc0106770 2024-04-26T21:06:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.148+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdda8004750 con 0x7fddc0106770 2024-04-26T21:06:55.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.150+0000 7fddbd600700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 0x7fdda40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:55.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.150+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5829+0+0 (secure 0 0 0) 0x7fdda80cfd50 con 0x7fddc0106770 2024-04-26T21:06:55.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.150+0000 7fddbfe00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 0x7fdda40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:55.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.151+0000 7fddbfe00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 0x7fdda40a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddb000f540 tx=0x7fddb000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.155+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdda809d040 con 0x7fddc0106770 2024-04-26T21:06:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:06:55.402+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fddc0063860 con 0x7fdda40a3cc0 2024-04-26T21:06:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:55 smithi027 ceph-mon[28085]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:55 smithi027 ceph-mon[28085]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:06:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:55 smithi204 ceph-mon[32145]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:55 smithi204 ceph-mon[32145]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:06:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:55.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:55 smithi094 ceph-mon[31273]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:55 smithi094 ceph-mon[31273]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:06:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:06:56.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:56 smithi027 ceph-mon[28085]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:56.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:56.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:56 smithi204 ceph-mon[32145]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:56.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:56 smithi094 ceph-mon[31273]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:06:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:06:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:57.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:57 smithi027 ceph-mon[28085]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:57.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:57 smithi204 ceph-mon[32145]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:57.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:57 smithi094 ceph-mon[31273]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:06:59.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:06:59 smithi027 ceph-mon[28085]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:06:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:06:59 smithi204 ceph-mon[32145]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:06:59.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:06:59 smithi094 ceph-mon[31273]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:07:00.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]: dispatch 2024-04-26T21:07:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]': finished 2024-04-26T21:07:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:00 smithi027 ceph-mon[28085]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:07:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]: dispatch 2024-04-26T21:07:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]': finished 2024-04-26T21:07:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:00 smithi204 ceph-mon[32145]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:07:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]: dispatch 2024-04-26T21:07:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3597381593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a783f5-a861-4ea2-b77f-f54e79b19da2"}]': finished 2024-04-26T21:07:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:00 smithi094 ceph-mon[31273]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:07:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:01.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:01 smithi027 ceph-mon[28085]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:07:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:01 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1780199646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:01 smithi204 ceph-mon[32145]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:07:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:01 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1780199646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:01.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:01 smithi094 ceph-mon[31273]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:07:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:01 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1780199646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:03.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:03 smithi027 ceph-mon[28085]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:07:03.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:03 smithi204 ceph-mon[32145]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:07:03.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:03 smithi094 ceph-mon[31273]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:07:05.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:05 smithi027 ceph-mon[28085]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T21:07:05.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:05 smithi204 ceph-mon[32145]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T21:07:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:05 smithi094 ceph-mon[31273]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T21:07:07.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:07 smithi027 ceph-mon[28085]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:07:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:07 smithi204 ceph-mon[32145]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:07:07.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:07 smithi094 ceph-mon[31273]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:07:09.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:09 smithi027 ceph-mon[28085]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:09 smithi204 ceph-mon[32145]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:09 smithi094 ceph-mon[31273]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:10 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:07:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:10 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:10.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:10 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:07:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:10 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:10.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:10 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:07:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:10 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:11.726 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:11 smithi027 ceph-mon[28085]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:11.727 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:11 smithi027 ceph-mon[28085]: Deploying daemon osd.9 on smithi027 2024-04-26T21:07:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:11 smithi204 ceph-mon[32145]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:11.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:11 smithi204 ceph-mon[32145]: Deploying daemon osd.9 on smithi027 2024-04-26T21:07:11.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:11 smithi094 ceph-mon[31273]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:11 smithi094 ceph-mon[31273]: Deploying daemon osd.9 on smithi027 2024-04-26T21:07:13.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:13 smithi027 ceph-mon[28085]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:13.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:13 smithi204 ceph-mon[32145]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:13.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:13 smithi094 ceph-mon[31273]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:15.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:15 smithi027 ceph-mon[28085]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:15 smithi204 ceph-mon[32145]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:15.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:15 smithi094 ceph-mon[31273]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:17 smithi204 ceph-mon[32145]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:17 smithi094 ceph-mon[31273]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:17.895 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:17 smithi027 ceph-mon[28085]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:19.533 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:19 smithi027 ceph-mon[28085]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:19 smithi204 ceph-mon[32145]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:19.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:19 smithi094 ceph-mon[31273]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:21.035 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.035 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.036 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:20 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:21.036 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:20 smithi027 ceph-mon[28085]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:21.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:20 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:21.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:20 smithi204 ceph-mon[32145]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:21.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:21.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:20 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:21.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:20 smithi094 ceph-mon[31273]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:23.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:23 smithi027 ceph-mon[28085]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:23.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:23 smithi204 ceph-mon[32145]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:23.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:23 smithi094 ceph-mon[31273]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:25.067 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-04-26T21:07:25.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.065+0000 7fddbd600700 1 -- 172.21.15.27:0/2877046964 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddc0063860 con 0x7fdda40a3cc0 2024-04-26T21:07:25.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.068+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 msgr2=0x7fdda40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:25.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.068+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 0x7fdda40a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddb000f540 tx=0x7fddb000b040 comp rx=0 tx=0).stop 2024-04-26T21:07:25.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.068+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 msgr2=0x7fddc01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.068+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e9c0 tx=0x7fdda800ecd0 comp rx=0 tx=0).stop 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 shutdown_connections 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdda40a3cc0 0x7fdda40a6180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01090a0 0x7fddc0194900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddc0103e40 0x7fddc0197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:25.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 --2- 172.21.15.27:0/2877046964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddc0106770 0x7fddc01982c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 >> 172.21.15.27:0/2877046964 conn(0x7fddc00fd830 msgr2=0x7fddc0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 shutdown_connections 2024-04-26T21:07:25.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:07:25.069+0000 7fddc5a00700 1 -- 172.21.15.27:0/2877046964 wait complete. 2024-04-26T21:07:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:24 smithi027 ceph-mon[28085]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:25.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:24 smithi027 ceph-mon[28085]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:07:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:24 smithi204 ceph-mon[32145]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:25.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:24 smithi204 ceph-mon[32145]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:07:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:24 smithi094 ceph-mon[31273]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:25.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:24 smithi094 ceph-mon[31273]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:07:25.923 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:07:25.923 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:07:25.923 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:07:25.923 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:25.924 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.924 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:25.990 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:07:25.995 INFO:tasks.cephadm:Deploying osd.10 on smithi094 with /dev/vg_nvme/lv_1... 2024-04-26T21:07:25.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:07:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:07:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:07:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:07:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:07:27.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-mon[28085]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:07:27.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-mon[28085]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:07:27.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-mon[28085]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:27.021 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:07:26 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:07:26.983+0000 7fa40d600700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T21:07:27.021 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:07:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:07:27.018+0000 7fa403400700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:07:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:26 smithi204 ceph-mon[32145]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:07:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:26 smithi204 ceph-mon[32145]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:07:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:26 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:26 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:26 smithi204 ceph-mon[32145]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:27.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:26 smithi094 ceph-mon[31273]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-26T21:07:27.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:26 smithi094 ceph-mon[31273]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:07:27.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:26 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:26 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:27.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:26 smithi094 ceph-mon[31273]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:28.149 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:27 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:07:28.149 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:27 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:07:28.149 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:27 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:28.149 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:27 smithi027 ceph-mon[28085]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' 2024-04-26T21:07:28.189 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:27 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:07:28.190 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:27 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:07:28.190 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:27 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:28.190 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:27 smithi094 ceph-mon[31273]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' 2024-04-26T21:07:28.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:27 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:07:28.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:27 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:07:28.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:27 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:28.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:27 smithi204 ceph-mon[32145]: from='osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206]' entity='osd.9' 2024-04-26T21:07:29.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:28 smithi027 ceph-mon[28085]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:29.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:28 smithi027 ceph-mon[28085]: osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] boot 2024-04-26T21:07:29.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:28 smithi027 ceph-mon[28085]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:07:29.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:28 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:28 smithi204 ceph-mon[32145]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:29.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:28 smithi204 ceph-mon[32145]: osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] boot 2024-04-26T21:07:29.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:28 smithi204 ceph-mon[32145]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:07:29.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:28 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:28 smithi094 ceph-mon[31273]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:07:29.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:28 smithi094 ceph-mon[31273]: osd.9 [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] boot 2024-04-26T21:07:29.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:28 smithi094 ceph-mon[31273]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:07:29.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:28 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:07:29.405 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:07:29.405 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:07:29.405 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-26T21:07:29.405 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-26T21:07:29.406 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0348661 s, 301 MB/s 2024-04-26T21:07:29.406 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-26T21:07:30.226 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:29 smithi094 ceph-mon[31273]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:07:30.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:29 smithi027 ceph-mon[28085]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:07:30.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:29 smithi204 ceph-mon[32145]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:07:30.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-04-26T21:07:30.776 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:07:31.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:31 smithi027 ceph-mon[28085]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:31.188 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:31 smithi094 ceph-mon[31273]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:31.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:31 smithi204 ceph-mon[32145]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi027 2024-04-26T21:07:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:32.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:32 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi027 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.512 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:32 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi027 2024-04-26T21:07:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 4272M 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:32.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:32 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 -- 172.21.15.94:0/3132530411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82481056c0 msgr2=0x7f8248105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 --2- 172.21.15.94:0/3132530411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82481056c0 0x7f8248105ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f823c0099c0 tx=0x7f823c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 -- 172.21.15.94:0/3132530411 shutdown_connections 2024-04-26T21:07:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 --2- 172.21.15.94:0/3132530411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8248107260 0x7f824810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 --2- 172.21.15.94:0/3132530411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82481056c0 0x7f8248105ae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 --2- 172.21.15.94:0/3132530411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 0x7f8248106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 -- 172.21.15.94:0/3132530411 >> 172.21.15.94:0/3132530411 conn(0x7f8248100c80 msgr2=0x7f82481030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:32.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 -- 172.21.15.94:0/3132530411 shutdown_connections 2024-04-26T21:07:32.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.864+0000 7f8250600700 1 -- 172.21.15.94:0/3132530411 wait complete. 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.865+0000 7f8250600700 1 Processor -- start 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 -- start start 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82481056c0 0x7f824819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 0x7f824819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8248111010 con 0x7f82481068c0 2024-04-26T21:07:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8248110e90 con 0x7f8248107260 2024-04-26T21:07:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f8250600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8248111190 con 0x7f82481056c0 2024-04-26T21:07:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f824e600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f824e600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38952/0 (socket says 172.21.15.94:38952) 2024-04-26T21:07:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.866+0000 7f824e600700 1 -- 172.21.15.94:0/333536485 learned_addr learned my addr 172.21.15.94:0/333536485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824dc00700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82481056c0 0x7f824819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824d200700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 0x7f824819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 -- 172.21.15.94:0/333536485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82481056c0 msgr2=0x7f824819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82481056c0 0x7f824819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 -- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 msgr2=0x7f824819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 0x7f824819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 -- 172.21.15.94:0/333536485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a3350 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f824e600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8240007ee0 tx=0x7f824000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000dbd0 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f824000dd70 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.867+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a3640 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.868+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400058d0 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.868+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a3b80 con 0x7f8248107260 2024-04-26T21:07:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.869+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824806a110 con 0x7f8248107260 2024-04-26T21:07:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.871+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f824001a020 con 0x7f8248107260 2024-04-26T21:07:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.872+0000 7f8246000700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.872+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7f82400cf4a0 con 0x7f8248107260 2024-04-26T21:07:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.873+0000 7f824dc00700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.874+0000 7f824dc00700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 0x7f82300a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f823c009990 tx=0x7f823c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:32.877+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8240010070 con 0x7f8248107260 2024-04-26T21:07:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:33.114+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8248063860 con 0x7f82300a3cc0 2024-04-26T21:07:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:33 smithi204 ceph-mon[32145]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:07:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:07:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:33.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:33 smithi094 ceph-mon[31273]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:33.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:07:33.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:07:33.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:33 smithi027 ceph-mon[28085]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:07:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:07:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:34.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:34 smithi094 ceph-mon[31273]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:07:34.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:34 smithi027 ceph-mon[28085]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:07:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:34 smithi204 ceph-mon[32145]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:07:35.513 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:35 smithi094 ceph-mon[31273]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:35.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:35 smithi027 ceph-mon[28085]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:35.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:35 smithi204 ceph-mon[32145]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:37.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/2557727395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]': finished 2024-04-26T21:07:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:07:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:07:37.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/2557727395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]': finished 2024-04-26T21:07:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:07:37.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/2557727395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]: dispatch 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70efbe0c-9374-43d6-b479-f1412debf3e7"}]': finished 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:07:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:07:38.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:38 smithi027 ceph-mon[28085]: from='client.? 172.21.15.94:0/635719092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:38 smithi204 ceph-mon[32145]: from='client.? 172.21.15.94:0/635719092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:38.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:38 smithi094 ceph-mon[31273]: from='client.? 172.21.15.94:0/635719092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:07:39.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:39 smithi027 ceph-mon[28085]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:39.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:39 smithi204 ceph-mon[32145]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:39.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:39 smithi094 ceph-mon[31273]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:41.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:41 smithi094 ceph-mon[31273]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:41.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:41 smithi204 ceph-mon[32145]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:41.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:41 smithi027 ceph-mon[28085]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:43.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:43 smithi094 ceph-mon[31273]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:43.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:43 smithi027 ceph-mon[28085]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:43.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:43 smithi204 ceph-mon[32145]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:45.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:45 smithi027 ceph-mon[28085]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:45 smithi204 ceph-mon[32145]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:45.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:45 smithi094 ceph-mon[31273]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:47.593 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:47 smithi094 ceph-mon[31273]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:47.594 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:07:47.594 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:47 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:47.594 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:47 smithi094 ceph-mon[31273]: Deploying daemon osd.10 on smithi094 2024-04-26T21:07:47.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:47 smithi027 ceph-mon[28085]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:47.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:07:47.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:47 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:47.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:47 smithi027 ceph-mon[28085]: Deploying daemon osd.10 on smithi094 2024-04-26T21:07:47.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:47 smithi204 ceph-mon[32145]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:07:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:47 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:47.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:47 smithi204 ceph-mon[32145]: Deploying daemon osd.10 on smithi094 2024-04-26T21:07:49.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:49 smithi094 ceph-mon[31273]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:49.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:49 smithi027 ceph-mon[28085]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:49.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:49 smithi204 ceph-mon[32145]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:51.486 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:51 smithi094 ceph-mon[31273]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:51.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:51 smithi027 ceph-mon[28085]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:51.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:51 smithi204 ceph-mon[32145]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:53.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:53 smithi027 ceph-mon[28085]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:53.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:53 smithi204 ceph-mon[32145]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:53.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:53 smithi094 ceph-mon[31273]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:55.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:55 smithi094 ceph-mon[31273]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:55.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:55 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:55.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:55 smithi027 ceph-mon[28085]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:55 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:55 smithi204 ceph-mon[32145]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:55 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:57.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:57 smithi094 ceph-mon[31273]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:57.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:57 smithi027 ceph-mon[28085]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:57.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:57 smithi204 ceph-mon[32145]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:59.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:07:59.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:07:59 smithi027 ceph-mon[28085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:07:59.370 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:07:59.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:07:59 smithi094 ceph-mon[31273]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:07:59.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.486+0000 7f8246000700 1 -- 172.21.15.94:0/333536485 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8248063860 con 0x7f82300a3cc0 2024-04-26T21:07:59.488 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 10 on host 'smithi094' 2024-04-26T21:07:59.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.489+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 msgr2=0x7f82300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:59.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.489+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 0x7f82300a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f823c009990 tx=0x7f823c01a040 comp rx=0 tx=0).stop 2024-04-26T21:07:59.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.489+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 msgr2=0x7f82481a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:59.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.489+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8240007ee0 tx=0x7f824000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 shutdown_connections 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8248107260 0x7f82481a2cb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82481056c0 0x7f824819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 --2- 172.21.15.94:0/333536485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481068c0 0x7f824819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 >> 172.21.15.94:0/333536485 conn(0x7f8248100c80 msgr2=0x7f8248109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 shutdown_connections 2024-04-26T21:07:59.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T21:07:59.490+0000 7f8250600700 1 -- 172.21.15.94:0/333536485 wait complete. 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:07:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:07:59 smithi204 ceph-mon[32145]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:08:00.242 DEBUG:teuthology.orchestra.run.smithi094:osd.10> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:08:00.246 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-04-26T21:08:00.247 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:08:00.491 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:00.492 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:08:00.366+0000 7f12ea400700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T21:08:00.492 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:08:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:08:00.392+0000 7f12e4800700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:00.686 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:08:00.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:08:00.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:00 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:00.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:08:00.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:08:00.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:00.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.756 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-26T21:08:00.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:00.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:00 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:01 smithi204 ceph-mon[32145]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:08:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:01 smithi204 ceph-mon[32145]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:08:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:01 smithi204 ceph-mon[32145]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:08:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:01.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:01.680 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:01 smithi094 ceph-mon[31273]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:08:01.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:01 smithi094 ceph-mon[31273]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:08:01.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:01 smithi094 ceph-mon[31273]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:08:01.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:01.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:01.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:01 smithi027 ceph-mon[28085]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:08:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:01 smithi027 ceph-mon[28085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-26T21:08:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:01 smithi027 ceph-mon[28085]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:08:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:02 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:08:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:02 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:08:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:02 smithi204 ceph-mon[32145]: osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] boot 2024-04-26T21:08:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:02 smithi204 ceph-mon[32145]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:08:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:02.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:02 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:08:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:02 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:08:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:02 smithi027 ceph-mon[28085]: osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] boot 2024-04-26T21:08:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:02 smithi027 ceph-mon[28085]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:08:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:02 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:08:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:02 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:08:02.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:02 smithi094 ceph-mon[31273]: osd.10 [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] boot 2024-04-26T21:08:02.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:02 smithi094 ceph-mon[31273]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:08:02.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:08:03.555 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:08:03.555 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:08:03.555 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-04-26T21:08:03.555 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-04-26T21:08:03.555 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0364561 s, 288 MB/s 2024-04-26T21:08:03.556 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-04-26T21:08:03.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:03 smithi027 ceph-mon[28085]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:03.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:03 smithi027 ceph-mon[28085]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:08:03.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:03 smithi204 ceph-mon[32145]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:03.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:03 smithi204 ceph-mon[32145]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:08:03.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:03 smithi094 ceph-mon[31273]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:03.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:03 smithi094 ceph-mon[31273]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:08:04.405 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-04-26T21:08:04.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:04 smithi204 ceph-mon[32145]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:08:04.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:04 smithi027 ceph-mon[28085]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:08:04.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:04 smithi094 ceph-mon[31273]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:08:04.817 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:05.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:05.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:05.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:05.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:06.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:06 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi094 2024-04-26T21:08:06.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:06 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T21:08:06.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:06 smithi204 ceph-mon[32145]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:08:06.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:06 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi094 2024-04-26T21:08:06.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:06 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T21:08:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:06 smithi027 ceph-mon[28085]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:08:06.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:06 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi094 2024-04-26T21:08:06.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:06 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-26T21:08:06.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:06 smithi094 ceph-mon[31273]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 -- 172.21.15.204:0/364350832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c109170 msgr2=0x7f830c10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 --2- 172.21.15.204:0/364350832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c109170 0x7f830c10b5e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f830400b0a0 tx=0x7f830400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 -- 172.21.15.204:0/364350832 shutdown_connections 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 --2- 172.21.15.204:0/364350832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c109170 0x7f830c10b5e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 --2- 172.21.15.204:0/364350832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 --2- 172.21.15.204:0/364350832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c106840 0x7f830c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 -- 172.21.15.204:0/364350832 >> 172.21.15.204:0/364350832 conn(0x7f830c0fd920 msgr2=0x7f830c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.829+0000 7f8312600700 1 -- 172.21.15.204:0/364350832 shutdown_connections 2024-04-26T21:08:06.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.830+0000 7f8312600700 1 -- 172.21.15.204:0/364350832 wait complete. 2024-04-26T21:08:06.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.831+0000 7f8312600700 1 Processor -- start 2024-04-26T21:08:06.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.831+0000 7f8312600700 1 -- start start 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 0x7f830c1a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10ede0 con 0x7f830c103f10 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10ec60 con 0x7f830c109170 2024-04-26T21:08:06.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f8312600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10ef60 con 0x7f830c106840 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f830aa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f830aa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:37572/0 (socket says 172.21.15.204:37572) 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f830aa00700 1 -- 172.21.15.204:0/631741970 learned_addr learned my addr 172.21.15.204:0/631741970 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.832+0000 7f830b400700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 -- 172.21.15.204:0/631741970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 msgr2=0x7f830c1a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830be00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 0x7f830c1a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 0x7f830c1a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 -- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 msgr2=0x7f830c19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c19b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 -- 172.21.15.204:0/631741970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1a32c0 con 0x7f830c106840 2024-04-26T21:08:06.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830b400700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830be00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 0x7f830c1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f830aa00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c9e0 tx=0x7f82f800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8007ac0 con 0x7f830c106840 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82f8007c60 con 0x7f830c106840 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1a35b0 con 0x7f830c106840 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8018660 con 0x7f830c106840 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.833+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a3af0 con 0x7f830c106840 2024-04-26T21:08:06.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.835+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c06a110 con 0x7f830c106840 2024-04-26T21:08:06.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.837+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f82f800f450 con 0x7f830c106840 2024-04-26T21:08:06.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.839+0000 7f8303e00700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:06.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.839+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6694+0+0 (secure 0 0 0) 0x7f82f80cfe50 con 0x7f830c106840 2024-04-26T21:08:06.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.839+0000 7f830b400700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:06.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.841+0000 7f830b400700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 0x7f82f00a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005fd0 tx=0x7f82fc005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:06.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:06.844+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82f8014030 con 0x7f830c106840 2024-04-26T21:08:07.079 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:07.078+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f830c063860 con 0x7f82f00a3c40 2024-04-26T21:08:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:07 smithi027 ceph-mon[28085]: pgmap v311: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:07 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:08:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:07 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:08:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:07 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:07 smithi204 ceph-mon[32145]: pgmap v311: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:07 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:08:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:07 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:08:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:07 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:07 smithi094 ceph-mon[31273]: pgmap v311: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:07 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:08:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:07 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:08:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:07 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:08.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:08 smithi027 ceph-mon[28085]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:08.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:08 smithi204 ceph-mon[32145]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:08.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:08 smithi094 ceph-mon[31273]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:09.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:09 smithi204 ceph-mon[32145]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T21:08:09.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:09 smithi027 ceph-mon[28085]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T21:08:09.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:09 smithi094 ceph-mon[31273]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/2898163805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]': finished 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:08:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/2898163805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]': finished 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:08:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:11.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-26T21:08:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/2898163805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]: dispatch 2024-04-26T21:08:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c543ea6d-1e49-42e8-aea5-1cea36e42415"}]': finished 2024-04-26T21:08:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:08:11.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:12.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:12 smithi027 ceph-mon[28085]: from='client.? 172.21.15.204:0/2790555588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:08:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:12 smithi204 ceph-mon[32145]: from='client.? 172.21.15.204:0/2790555588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:08:12.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:12 smithi094 ceph-mon[31273]: from='client.? 172.21.15.204:0/2790555588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:08:13.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:13 smithi027 ceph-mon[28085]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:08:13.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:13 smithi204 ceph-mon[32145]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:08:13.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:13 smithi094 ceph-mon[31273]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:08:15.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:15 smithi027 ceph-mon[28085]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:08:15.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:15 smithi204 ceph-mon[32145]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:08:15.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:15 smithi094 ceph-mon[31273]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:08:17.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:17 smithi204 ceph-mon[32145]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:08:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:17 smithi094 ceph-mon[31273]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:08:17.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:17 smithi027 ceph-mon[28085]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:08:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:19 smithi204 ceph-mon[32145]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:19.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:19 smithi094 ceph-mon[31273]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:19.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:19 smithi027 ceph-mon[28085]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:21 smithi204 ceph-mon[32145]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:08:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:21.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:21 smithi204 ceph-mon[32145]: Deploying daemon osd.11 on smithi204 2024-04-26T21:08:21.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:21 smithi094 ceph-mon[31273]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:21.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:08:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:21 smithi094 ceph-mon[31273]: Deploying daemon osd.11 on smithi204 2024-04-26T21:08:21.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:21 smithi027 ceph-mon[28085]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:08:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:21 smithi027 ceph-mon[28085]: Deploying daemon osd.11 on smithi204 2024-04-26T21:08:23.719 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:23 smithi204 ceph-mon[32145]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:23.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:23 smithi094 ceph-mon[31273]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:23.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:23 smithi027 ceph-mon[28085]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:25 smithi094 ceph-mon[31273]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:25.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:25 smithi204 ceph-mon[32145]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:25.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:25 smithi027 ceph-mon[28085]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:27 smithi204 ceph-mon[32145]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:27.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:27 smithi094 ceph-mon[31273]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:27.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:27 smithi027 ceph-mon[28085]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:29.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:29 smithi204 ceph-mon[32145]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:29.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:29.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:29.930 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:29.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:29 smithi027 ceph-mon[28085]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:29.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:29.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:29.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:30.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:29 smithi094 ceph-mon[31273]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:30.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:30.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:30.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:31 smithi027 ceph-mon[28085]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:32.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:31 smithi204 ceph-mon[32145]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:32.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:31 smithi094 ceph-mon[31273]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:33.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.308+0000 7f8303e00700 1 -- 172.21.15.204:0/631741970 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f830c063860 con 0x7f82f00a3c40 2024-04-26T21:08:33.311 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-04-26T21:08:33.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 msgr2=0x7f82f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 0x7f82f00a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005fd0 tx=0x7f82fc005e20 comp rx=0 tx=0).stop 2024-04-26T21:08:33.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 msgr2=0x7f830c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c9e0 tx=0x7f82f800cda0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 shutdown_connections 2024-04-26T21:08:33.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.314 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c109170 0x7f830c1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830c103f10 0x7f830c19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 --2- 172.21.15.204:0/631741970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f830c106840 0x7f830c19bba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.312+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 >> 172.21.15.204:0/631741970 conn(0x7f830c0fd920 msgr2=0x7f830c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:33.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.313+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 shutdown_connections 2024-04-26T21:08:33.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:33.313+0000 7f8312600700 1 -- 172.21.15.204:0/631741970 wait complete. 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:33.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:33.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:33.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:34.133 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:08:34.139 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:08:34.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:08:34.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:34 smithi027 ceph-mon[28085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:08:34.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:34 smithi027 ceph-mon[28085]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:34 smithi027 ceph-mon[28085]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:08:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:34 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:34 smithi027 ceph-mon[28085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:34.617 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:08:34.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:34 smithi204 ceph-mon[32145]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:08:34.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:34 smithi204 ceph-mon[32145]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:34.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:34 smithi204 ceph-mon[32145]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:08:34.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:34 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:34.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:34 smithi204 ceph-mon[32145]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:34.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:34 smithi094 ceph-mon[31273]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:08:34.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:34 smithi094 ceph-mon[31273]: from='osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:34.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:34 smithi094 ceph-mon[31273]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:08:34.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:34 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:34.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:34 smithi094 ceph-mon[31273]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-04-26T21:08:35.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-mon[32145]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:35.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-mon[32145]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:08:35.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-mon[32145]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:08:35.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:35.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:35.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:08:35.450+0000 7fc141000700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:08:35.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:08:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:08:35.491+0000 7fc136e00700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:35.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:35 smithi027 ceph-mon[28085]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:35.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:35 smithi027 ceph-mon[28085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:08:35.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:35 smithi027 ceph-mon[28085]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:08:35.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:35.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:35 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:35.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:35 smithi094 ceph-mon[31273]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:35.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:35 smithi094 ceph-mon[31273]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-04-26T21:08:35.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:35 smithi094 ceph-mon[31273]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:08:35.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:35.673 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:35 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:36.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:36 smithi027 ceph-mon[28085]: purged_snaps scrub starts 2024-04-26T21:08:36.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:36 smithi027 ceph-mon[28085]: purged_snaps scrub ok 2024-04-26T21:08:36.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:36 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:36.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:36 smithi027 ceph-mon[28085]: from='osd.11 ' entity='osd.11' 2024-04-26T21:08:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:36 smithi204 ceph-mon[32145]: purged_snaps scrub starts 2024-04-26T21:08:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:36 smithi204 ceph-mon[32145]: purged_snaps scrub ok 2024-04-26T21:08:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:36 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:36 smithi204 ceph-mon[32145]: from='osd.11 ' entity='osd.11' 2024-04-26T21:08:36.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:36 smithi094 ceph-mon[31273]: purged_snaps scrub starts 2024-04-26T21:08:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:36 smithi094 ceph-mon[31273]: purged_snaps scrub ok 2024-04-26T21:08:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:36 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:36 smithi094 ceph-mon[31273]: from='osd.11 ' entity='osd.11' 2024-04-26T21:08:37.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 -- 172.21.15.27:0/4217366437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1034d0 msgr2=0x7f316c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 --2- 172.21.15.27:0/4217366437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1034d0 0x7f316c1038f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f315c0099c0 tx=0x7f315c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 -- 172.21.15.27:0/4217366437 shutdown_connections 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 --2- 172.21.15.27:0/4217366437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c105070 0x7f316c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 --2- 172.21.15.27:0/4217366437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1034d0 0x7f316c1038f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 --2- 172.21.15.27:0/4217366437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c1046d0 0x7f316c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 -- 172.21.15.27:0/4217366437 >> 172.21.15.27:0/4217366437 conn(0x7f316c0fea70 msgr2=0x7f316c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 -- 172.21.15.27:0/4217366437 shutdown_connections 2024-04-26T21:08:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.011+0000 7f3173200700 1 -- 172.21.15.27:0/4217366437 wait complete. 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.012+0000 7f3173200700 1 Processor -- start 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.012+0000 7f3173200700 1 -- start start 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c1034d0 0x7f316c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 0x7f316c1a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10ee60 con 0x7f316c1046d0 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10ece0 con 0x7f316c1034d0 2024-04-26T21:08:37.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3173200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10efe0 con 0x7f316c105070 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f316b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f316b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40504/0 (socket says 172.21.15.27:40504) 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f316b400700 1 -- 172.21.15.27:0/332144963 learned_addr learned my addr 172.21.15.27:0/332144963 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f3171200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 0x7f316c1a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.013+0000 7f316be00700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c1034d0 0x7f316c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 -- 172.21.15.27:0/332144963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 msgr2=0x7f316c1a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 0x7f316c1a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 -- 172.21.15.27:0/332144963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c1034d0 msgr2=0x7f316c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c1034d0 0x7f316c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 -- 172.21.15.27:0/332144963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c1a3380 con 0x7f316c1046d0 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f3171200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 0x7f316c1a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.014+0000 7f316b400700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9d0 tx=0x7f315800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.015+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315800cbd0 con 0x7f316c1046d0 2024-04-26T21:08:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.015+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315800ee70 con 0x7f316c1046d0 2024-04-26T21:08:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.015+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158018680 con 0x7f316c1046d0 2024-04-26T21:08:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.015+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c1a3670 con 0x7f316c1046d0 2024-04-26T21:08:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.015+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a3b60 con 0x7f316c1046d0 2024-04-26T21:08:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.016+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c06a110 con 0x7f316c1046d0 2024-04-26T21:08:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.019+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3158010430 con 0x7f316c1046d0 2024-04-26T21:08:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.020+0000 7f3168c00700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 0x7f31500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.021+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f31580d0550 con 0x7f316c1046d0 2024-04-26T21:08:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.021+0000 7f316be00700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 0x7f31500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.022+0000 7f316be00700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 0x7f31500a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b580 tx=0x7f315c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:37.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.023+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f315809d330 con 0x7f316c1046d0 2024-04-26T21:08:37.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.259+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f316c1a3fa0 con 0x7f316c1046d0 2024-04-26T21:08:37.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.261+0000 7f3168c00700 1 -- 172.21.15.27:0/332144963 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f315809cc30 con 0x7f316c1046d0 2024-04-26T21:08:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:37.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 msgr2=0x7f31500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 0x7f31500a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b580 tx=0x7f315c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 msgr2=0x7f316c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9d0 tx=0x7f315800ece0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 shutdown_connections 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31500a3cc0 0x7f31500a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c105070 0x7f316c1a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f316c1034d0 0x7f316c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 --2- 172.21.15.27:0/332144963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f316c1046d0 0x7f316c19bcb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.264+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 >> 172.21.15.27:0/332144963 conn(0x7f316c0fea70 msgr2=0x7f316c109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.265+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 shutdown_connections 2024-04-26T21:08:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:37.265+0000 7f3173200700 1 -- 172.21.15.27:0/332144963 wait complete. 2024-04-26T21:08:37.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:37.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] boot 2024-04-26T21:08:37.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:08:37.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:37 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/332144963' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] boot 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:37 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/332144963' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: osd.11 [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] boot 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:08:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:37 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/332144963' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:08:38.193 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714165716,"num_in_osds":12,"osd_in_since":1714165690,"num_remapped_pgs":0} 2024-04-26T21:08:38.194 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:08:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:08:38.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:38 smithi027 ceph-mon[28085]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:08:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:38 smithi204 ceph-mon[32145]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:08:38.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:38 smithi094 ceph-mon[31273]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:08:39.811 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:39.811 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: Detected new or changed devices on smithi204 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 4272M 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:39.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:39 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: Detected new or changed devices on smithi204 2024-04-26T21:08:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 4272M 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:40.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:39 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: Detected new or changed devices on smithi204 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 4272M 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:40.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:39 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:40.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 -- 172.21.15.27:0/4183533201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c9010b620 msgr2=0x7f7c9010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 --2- 172.21.15.27:0/4183533201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c9010b620 0x7f7c9010ba00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c800099c0 tx=0x7f7c80009cd0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 -- 172.21.15.27:0/4183533201 shutdown_connections 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 --2- 172.21.15.27:0/4183533201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90105f30 0x7f7c901123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 --2- 172.21.15.27:0/4183533201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c9010b620 0x7f7c9010ba00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 --2- 172.21.15.27:0/4183533201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105570 0x7f7c901059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 -- 172.21.15.27:0/4183533201 >> 172.21.15.27:0/4183533201 conn(0x7f7c90101040 msgr2=0x7f7c90103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 -- 172.21.15.27:0/4183533201 shutdown_connections 2024-04-26T21:08:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.833+0000 7f7c95600700 1 -- 172.21.15.27:0/4183533201 wait complete. 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.834+0000 7f7c95600700 1 Processor -- start 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.834+0000 7f7c95600700 1 -- start start 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 0x7f7c901a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 0x7f7c9019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90115640 con 0x7f7c90105570 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c901154c0 con 0x7f7c9010b620 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c95600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c901157c0 con 0x7f7c90105f30 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58696/0 (socket says 172.21.15.27:58696) 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 -- 172.21.15.27:0/2551124326 learned_addr learned my addr 172.21.15.27:0/2551124326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8f400700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 0x7f7c901a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 -- 172.21.15.27:0/2551124326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 msgr2=0x7f7c901a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 0x7f7c901a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c94c00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 0x7f7c9019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 -- 172.21.15.27:0/2551124326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 msgr2=0x7f7c9019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 0x7f7c9019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8fe00700 1 -- 172.21.15.27:0/2551124326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9019c0c0 con 0x7f7c90105570 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.835+0000 7f7c8f400700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 0x7f7c901a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c94c00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 0x7f7c9019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c8fe00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000b580 tx=0x7f7c8000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80027070 con 0x7f7c90105570 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9019c350 con 0x7f7c90105570 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c8000fdd0 con 0x7f7c90105570 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800173f0 con 0x7f7c90105570 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.836+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c901a8620 con 0x7f7c90105570 2024-04-26T21:08:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.838+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9010dab0 con 0x7f7c90105570 2024-04-26T21:08:40.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.840+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7c8002a070 con 0x7f7c90105570 2024-04-26T21:08:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.841+0000 7f7c8cc00700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.841+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7c800d0f90 con 0x7f7c90105570 2024-04-26T21:08:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.841+0000 7f7c8f400700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.842+0000 7f7c8f400700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 0x7f7c740a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005fd0 tx=0x7f7c7c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:40.846+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c8005f060 con 0x7f7c90105570 2024-04-26T21:08:41.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.078+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c9019ce40 con 0x7f7c90105570 2024-04-26T21:08:41.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.080+0000 7f7c8cc00700 1 -- 172.21.15.27:0/2551124326 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7f7c80013070 con 0x7f7c90105570 2024-04-26T21:08:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:41.082 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:08:37.469170+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 msgr2=0x7f7c740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 0x7f7c740a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005fd0 tx=0x7f7c7c005ee0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 msgr2=0x7f7c901a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000b580 tx=0x7f7c8000f510 comp rx=0 tx=0).stop 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 shutdown_connections 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c9010b620 0x7f7c9019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c90105570 0x7f7c901a1750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 --2- 172.21.15.27:0/2551124326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c90105f30 0x7f7c901a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 >> 172.21.15.27:0/2551124326 conn(0x7f7c90101040 msgr2=0x7f7c90102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 shutdown_connections 2024-04-26T21:08:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:41.083+0000 7f7c95600700 1 -- 172.21.15.27:0/2551124326 wait complete. 2024-04-26T21:08:41.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:41 smithi027 ceph-mon[28085]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:41 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2551124326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:41.973 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:08:41.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:08:42.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:41 smithi204 ceph-mon[32145]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:42.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:41 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2551124326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:41 smithi094 ceph-mon[31273]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:41 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2551124326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:08:43.673 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:43 smithi027 ceph-mon[28085]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:44.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:43 smithi204 ceph-mon[32145]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:44.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:43 smithi094 ceph-mon[31273]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:44.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 -- 172.21.15.27:0/4240713309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081083b0 msgr2=0x7fda08108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 --2- 172.21.15.27:0/4240713309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081083b0 0x7fda08108790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80099c0 tx=0x7fd9f8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 -- 172.21.15.27:0/4240713309 shutdown_connections 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 --2- 172.21.15.27:0/4240713309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda08103e90 0x7fda08110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 --2- 172.21.15.27:0/4240713309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081083b0 0x7fda08108790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 --2- 172.21.15.27:0/4240713309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda081034d0 0x7fda08103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.720+0000 7fda0d000700 1 -- 172.21.15.27:0/4240713309 >> 172.21.15.27:0/4240713309 conn(0x7fda0807a580 msgr2=0x7fda0807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.721+0000 7fda0d000700 1 -- 172.21.15.27:0/4240713309 shutdown_connections 2024-04-26T21:08:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.721+0000 7fda0d000700 1 -- 172.21.15.27:0/4240713309 wait complete. 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 Processor -- start 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 -- start start 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda08103e90 0x7fda08075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 0x7fda080761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda081134f0 con 0x7fda081034d0 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.722+0000 7fda0d000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda08113370 con 0x7fda081083b0 2024-04-26T21:08:44.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.723+0000 7fda0d000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda08113670 con 0x7fda08103e90 2024-04-26T21:08:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.723+0000 7fda05e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.723+0000 7fda05e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58702/0 (socket says 172.21.15.27:58702) 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.723+0000 7fda05e00700 1 -- 172.21.15.27:0/3972051614 learned_addr learned my addr 172.21.15.27:0/3972051614 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.723+0000 7fda05400700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda08103e90 0x7fda08075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda06800700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 0x7fda080761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 -- 172.21.15.27:0/3972051614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda08103e90 msgr2=0x7fda08075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda08103e90 0x7fda08075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 -- 172.21.15.27:0/3972051614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 msgr2=0x7fda080761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 0x7fda080761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 -- 172.21.15.27:0/3972051614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda081a7f50 con 0x7fda081034d0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda06800700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 0x7fda080761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda05e00700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005f50 tx=0x7fd9f80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8027070 con 0x7fda081034d0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9f800fd60 con 0x7fda081034d0 2024-04-26T21:08:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081a81e0 con 0x7fda081034d0 2024-04-26T21:08:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.724+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8022e30 con 0x7fda081034d0 2024-04-26T21:08:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.725+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda081a8690 con 0x7fda081034d0 2024-04-26T21:08:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.726+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0806a110 con 0x7fda081034d0 2024-04-26T21:08:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.728+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9f802a070 con 0x7fda081034d0 2024-04-26T21:08:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.730+0000 7fd9fea00700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 0x7fd9ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.730+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd9f80d0cb0 con 0x7fda081034d0 2024-04-26T21:08:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.730+0000 7fda05400700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 0x7fd9ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.731+0000 7fda05400700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 0x7fd9ec0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda0807ac60 tx=0x7fd9f400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.733+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd9f8013070 con 0x7fda081034d0 2024-04-26T21:08:44.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.960+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda0802e650 con 0x7fda081034d0 2024-04-26T21:08:44.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.962+0000 7fd9fea00700 1 -- 172.21.15.27:0/3972051614 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fd9f809db20 con 0x7fda081034d0 2024-04-26T21:08:44.963 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.964+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 msgr2=0x7fd9ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.964+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 0x7fd9ec0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda0807ac60 tx=0x7fd9f400b3c0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.964+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 msgr2=0x7fda08075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.964+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005f50 tx=0x7fd9f80057c0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 shutdown_connections 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9ec0a3cc0 0x7fd9ec0a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda081083b0 0x7fda080761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda081034d0 0x7fda08075720 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 --2- 172.21.15.27:0/3972051614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda08103e90 0x7fda08075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 >> 172.21.15.27:0/3972051614 conn(0x7fda0807a580 msgr2=0x7fda08101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 shutdown_connections 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:44.965+0000 7fda0d000700 1 -- 172.21.15.27:0/3972051614 wait complete. 2024-04-26T21:08:45.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:45 smithi027 ceph-mon[28085]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:45.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:45 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3972051614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:45.887 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.b on smithi094 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.e on smithi094 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.h on smithi094 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.k on smithi094 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-04-26T21:08:45.888 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-04-26T21:08:45.888 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-04-26T21:08:45.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:45 smithi204 ceph-mon[32145]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:45.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:45 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3972051614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:46.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:45 smithi094 ceph-mon[31273]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:46.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:45 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3972051614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:46.335 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:08:47.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:47 smithi204 ceph-mon[32145]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:47 smithi027 ceph-mon[28085]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:47 smithi094 ceph-mon[31273]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.319+0000 7fe267200700 1 -- 172.21.15.204:0/3141407640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268106760 msgr2=0x7fe268108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.319+0000 7fe267200700 1 --2- 172.21.15.204:0/3141407640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268106760 0x7fe268108b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe254009990 tx=0x7fe254009ca0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 -- 172.21.15.204:0/3141407640 shutdown_connections 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 --2- 172.21.15.204:0/3141407640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268109090 0x7fe26810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 --2- 172.21.15.204:0/3141407640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 0x7fe268106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 --2- 172.21.15.204:0/3141407640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268106760 0x7fe268108b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 -- 172.21.15.204:0/3141407640 >> 172.21.15.204:0/3141407640 conn(0x7fe2680fd840 msgr2=0x7fe2680ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 -- 172.21.15.204:0/3141407640 shutdown_connections 2024-04-26T21:08:48.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.320+0000 7fe267200700 1 -- 172.21.15.204:0/3141407640 wait complete. 2024-04-26T21:08:48.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 Processor -- start 2024-04-26T21:08:48.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 -- start start 2024-04-26T21:08:48.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 0x7fe2681993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268106760 0x7fe268199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810ed20 con 0x7fe268103e30 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810eba0 con 0x7fe268106760 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe267200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810eea0 con 0x7fe268109090 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.321+0000 7fe266800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:49570/0 (socket says 172.21.15.204:49570) 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 -- 172.21.15.204:0/1332749498 learned_addr learned my addr 172.21.15.204:0/1332749498 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe265e00700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 0x7fe2681993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe265400700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268106760 0x7fe268199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 -- 172.21.15.204:0/1332749498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268106760 msgr2=0x7fe268199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268106760 0x7fe268199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 -- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 msgr2=0x7fe2681993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 0x7fe2681993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 -- 172.21.15.204:0/1332749498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2681a1050 con 0x7fe268109090 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe266800700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe260007ee0 tx=0x7fe26000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.322+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260004db0 con 0x7fe268109090 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.323+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe26000dbd0 con 0x7fe268109090 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.323+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2681a1340 con 0x7fe268109090 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.323+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260015400 con 0x7fe268109090 2024-04-26T21:08:48.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.323+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2681a1830 con 0x7fe268109090 2024-04-26T21:08:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.324+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26806a110 con 0x7fe268109090 2024-04-26T21:08:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.326+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe260005020 con 0x7fe268109090 2024-04-26T21:08:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.327+0000 7fe25ea00700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 0x7fe24c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.327+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe2600cfb40 con 0x7fe268109090 2024-04-26T21:08:48.329 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.328+0000 7fe265e00700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 0x7fe24c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.330 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.329+0000 7fe265e00700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 0x7fe24c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe258005950 tx=0x7fe2580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.332 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.332+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe260010070 con 0x7fe268109090 2024-04-26T21:08:48.572 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.570+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7fe268063b20 con 0x7fe24c0a3c40 2024-04-26T21:08:48.589 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.587+0000 7fe25ea00700 1 -- 172.21.15.204:0/1332749498 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe268063b20 con 0x7fe24c0a3c40 2024-04-26T21:08:48.589 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 msgr2=0x7fe24c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 0x7fe24c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe258005950 tx=0x7fe2580058e0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 msgr2=0x7fe2681a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe260007ee0 tx=0x7fe26000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 shutdown_connections 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe24c0a3c40 0x7fe24c0a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe268109090 0x7fe2681a09b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe268103e30 0x7fe2681993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 --2- 172.21.15.204:0/1332749498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe268106760 0x7fe268199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 >> 172.21.15.204:0/1332749498 conn(0x7fe2680fd840 msgr2=0x7fe268105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 shutdown_connections 2024-04-26T21:08:48.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-26T21:08:48.591+0000 7fe267200700 1 -- 172.21.15.204:0/1332749498 wait complete. 2024-04-26T21:08:49.308 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:08:49.316 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:08:49.363 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:08:49.409 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:08:49.456 DEBUG:teuthology.orchestra.run.smithi094:mds.b> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:08:49.464 DEBUG:teuthology.orchestra.run.smithi094:mds.e> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:08:49.511 DEBUG:teuthology.orchestra.run.smithi094:mds.h> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:08:49.558 DEBUG:teuthology.orchestra.run.smithi094:mds.k> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:08:49.603 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:08:49.610 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:08:49.655 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:08:49.702 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:49.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:49.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:49.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:49.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:49.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:49 smithi027 ceph-mon[28085]: Deploying daemon mds.l on smithi204 2024-04-26T21:08:49.746 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:08:49.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:49.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:49.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:49.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-26T21:08:49.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:49.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:49 smithi204 ceph-mon[32145]: Deploying daemon mds.l on smithi204 2024-04-26T21:08:49.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 -- 172.21.15.27:0/367232654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 msgr2=0x7f276810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 --2- 172.21.15.27:0/367232654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276810a400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2750009900 tx=0x7f275002a410 comp rx=0 tx=0).stop 2024-04-26T21:08:49.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 -- 172.21.15.27:0/367232654 shutdown_connections 2024-04-26T21:08:49.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 --2- 172.21.15.27:0/367232654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 0x7f27681111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 --2- 172.21.15.27:0/367232654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276810a400 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 --2- 172.21.15.27:0/367232654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 0x7f27681091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 -- 172.21.15.27:0/367232654 >> 172.21.15.27:0/367232654 conn(0x7f2768077d30 msgr2=0x7f276807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:49.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 -- 172.21.15.27:0/367232654 shutdown_connections 2024-04-26T21:08:49.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.991+0000 7f276d400640 1 -- 172.21.15.27:0/367232654 wait complete. 2024-04-26T21:08:49.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 Processor -- start 2024-04-26T21:08:49.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 -- start start 2024-04-26T21:08:49.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 0x7f276819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 0x7f27681a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27681147c0 con 0x7f2768109f80 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2768114640 con 0x7f276810a940 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.992+0000 7f276d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2768114940 con 0x7f2768108d90 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37560/0 (socket says 172.21.15.27:37560) 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 -- 172.21.15.27:0/623100838 learned_addr learned my addr 172.21.15.27:0/623100838 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f2767e00640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 0x7f276819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f276ca00640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 0x7f27681a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 -- 172.21.15.27:0/623100838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 msgr2=0x7f276819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 0x7f276819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 -- 172.21.15.27:0/623100838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 msgr2=0x7f27681a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 0x7f27681a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.993+0000 7f275f400640 1 -- 172.21.15.27:0/623100838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27681a6b40 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f275f400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2750031460 tx=0x7f2750003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275003f070 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2750031da0 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750035030 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27681a6dd0 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.994+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27681a7280 con 0x7f2768109f80 2024-04-26T21:08:49.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.995+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27681101d0 con 0x7f2768109f80 2024-04-26T21:08:49.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:49.997+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2750042070 con 0x7f2768109f80 2024-04-26T21:08:50.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.000+0000 7f2765600640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 0x7f274c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.000+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f27500f0900 con 0x7f2768109f80 2024-04-26T21:08:50.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.000+0000 7f2767e00640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 0x7f274c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.001+0000 7f2767e00640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 0x7f274c0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2758005fd0 tx=0x7f2758005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.005+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f27500bd6e0 con 0x7f2768109f80 2024-04-26T21:08:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:50.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:49 smithi094 ceph-mon[31273]: Deploying daemon mds.l on smithi204 2024-04-26T21:08:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.213+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27680635d0 con 0x7f2768109f80 2024-04-26T21:08:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.215+0000 7f2765600640 1 -- 172.21.15.27:0/623100838 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7f27500bcfe0 con 0x7f2768109f80 2024-04-26T21:08:50.217 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:08:37.469170+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 msgr2=0x7f274c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 0x7f274c0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2758005fd0 tx=0x7f2758005850 comp rx=0 tx=0).stop 2024-04-26T21:08:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 msgr2=0x7f276819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2750031460 tx=0x7f2750003c80 comp rx=0 tx=0).stop 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 shutdown_connections 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f274c0a3c30 0x7f274c0a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f276810a940 0x7f27681a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2768109f80 0x7f276819f460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 --2- 172.21.15.27:0/623100838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2768108d90 0x7f276819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 >> 172.21.15.27:0/623100838 conn(0x7f2768077d30 msgr2=0x7f276810d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 shutdown_connections 2024-04-26T21:08:50.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.218+0000 7f276d400640 1 -- 172.21.15.27:0/623100838 wait complete. 2024-04-26T21:08:50.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:08:50.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 -- 172.21.15.27:0/1430440337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078530 msgr2=0x7fbd48078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/1430440337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078530 0x7fbd48078990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009900 tx=0x7fbd3402a410 comp rx=0 tx=0).stop 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 -- 172.21.15.27:0/1430440337 shutdown_connections 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/1430440337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078ed0 0x7fbd48079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/1430440337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078530 0x7fbd48078990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/1430440337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 0x7fbd48077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.455+0000 7fbd4ce00640 1 -- 172.21.15.27:0/1430440337 >> 172.21.15.27:0/1430440337 conn(0x7fbd480fffe0 msgr2=0x7fbd48102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.456+0000 7fbd4ce00640 1 -- 172.21.15.27:0/1430440337 shutdown_connections 2024-04-26T21:08:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.456+0000 7fbd4ce00640 1 -- 172.21.15.27:0/1430440337 wait complete. 2024-04-26T21:08:50.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.457+0000 7fbd4ce00640 1 Processor -- start 2024-04-26T21:08:50.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.457+0000 7fbd4ce00640 1 -- start start 2024-04-26T21:08:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 0x7fbd4819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078530 0x7fbd4819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481148a0 con 0x7fbd48078ed0 2024-04-26T21:08:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48114720 con 0x7fbd48078530 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd4ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48114a20 con 0x7fbd480772f0 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd47e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd47e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37566/0 (socket says 172.21.15.27:37566) 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd47e00640 1 -- 172.21.15.27:0/972468532 learned_addr learned my addr 172.21.15.27:0/972468532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd47400640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 0x7fbd4819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.458+0000 7fbd46a00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078530 0x7fbd4819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 -- 172.21.15.27:0/972468532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 msgr2=0x7fbd4819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 0x7fbd4819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 -- 172.21.15.27:0/972468532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078530 msgr2=0x7fbd4819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078530 0x7fbd4819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 -- 172.21.15.27:0/972468532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481a6bd0 con 0x7fbd48078ed0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd47e00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000eaa0 tx=0x7fbd4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000cdb0 con 0x7fbd48078ed0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd40005020 con 0x7fbd48078ed0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400116d0 con 0x7fbd48078ed0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.459+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481a6ec0 con 0x7fbd48078ed0 2024-04-26T21:08:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.460+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a7330 con 0x7fbd48078ed0 2024-04-26T21:08:50.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.461+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd48111630 con 0x7fbd48078ed0 2024-04-26T21:08:50.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.463+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd400051d0 con 0x7fbd48078ed0 2024-04-26T21:08:50.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.464+0000 7fbd3fe00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 0x7fbd2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.465+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fbd400d0110 con 0x7fbd48078ed0 2024-04-26T21:08:50.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.465+0000 7fbd47400640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 0x7fbd2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.465+0000 7fbd47400640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 0x7fbd2c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd38005fd0 tx=0x7fbd3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.469+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbd40016030 con 0x7fbd48078ed0 2024-04-26T21:08:50.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.678+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd48063530 con 0x7fbd48078ed0 2024-04-26T21:08:50.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.679+0000 7fbd3fe00640 1 -- 172.21.15.27:0/972468532 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fbd4009c8a0 con 0x7fbd48078ed0 2024-04-26T21:08:50.681 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 msgr2=0x7fbd2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 0x7fbd2c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd38005fd0 tx=0x7fbd3800b390 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 msgr2=0x7fbd481a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000eaa0 tx=0x7fbd4000ef60 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 shutdown_connections 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbd2c0a3bb0 0x7fbd2c0a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078ed0 0x7fbd481a6560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd48078530 0x7fbd4819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 --2- 172.21.15.27:0/972468532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd480772f0 0x7fbd4819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 >> 172.21.15.27:0/972468532 conn(0x7fbd480fffe0 msgr2=0x7fbd48101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.682+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 shutdown_connections 2024-04-26T21:08:50.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.683+0000 7fbd4ce00640 1 -- 172.21.15.27:0/972468532 wait complete. 2024-04-26T21:08:50.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:50 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/623100838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 -- 172.21.15.27:0/3750664035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc0105d20 msgr2=0x7f7bc0108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 --2- 172.21.15.27:0/3750664035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc0105d20 0x7f7bc0108110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009900 tx=0x7f7ba802a410 comp rx=0 tx=0).stop 2024-04-26T21:08:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 -- 172.21.15.27:0/3750664035 shutdown_connections 2024-04-26T21:08:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 --2- 172.21.15.27:0/3750664035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 0x7f7bc010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 --2- 172.21.15.27:0/3750664035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc0105d20 0x7f7bc0108110 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 --2- 172.21.15.27:0/3750664035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc01033f0 0x7f7bc01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 -- 172.21.15.27:0/3750664035 >> 172.21.15.27:0/3750664035 conn(0x7f7bc00fcde0 msgr2=0x7f7bc00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 -- 172.21.15.27:0/3750664035 shutdown_connections 2024-04-26T21:08:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.916+0000 7f7bc5600640 1 -- 172.21.15.27:0/3750664035 wait complete. 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.917+0000 7f7bc5600640 1 Processor -- start 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.917+0000 7f7bc5600640 1 -- start start 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc0105d20 0x7f7bc019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 0x7f7bc01a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010e360 con 0x7f7bc01033f0 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010e1e0 con 0x7f7bc0108650 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bc5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010e4e0 con 0x7f7bc0105d20 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bbea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37590/0 (socket says 172.21.15.27:37590) 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bbea00640 1 -- 172.21.15.27:0/1414673990 learned_addr learned my addr 172.21.15.27:0/1414673990 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bbf400640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 0x7f7bc01a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.918+0000 7f7bb6000640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc0105d20 0x7f7bc019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 -- 172.21.15.27:0/1414673990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc0105d20 msgr2=0x7f7bc019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc0105d20 0x7f7bc019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 -- 172.21.15.27:0/1414673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 msgr2=0x7f7bc01a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 0x7f7bc01a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 -- 172.21.15.27:0/1414673990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc01a2970 con 0x7f7bc01033f0 2024-04-26T21:08:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bbea00640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000d770 tx=0x7f7bb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.919+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0017070 con 0x7f7bc01033f0 2024-04-26T21:08:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.920+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc01a2c60 con 0x7f7bc01033f0 2024-04-26T21:08:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.920+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bb00054b0 con 0x7f7bc01033f0 2024-04-26T21:08:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.920+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0020060 con 0x7f7bc01033f0 2024-04-26T21:08:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.920+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc01a3130 con 0x7f7bc01033f0 2024-04-26T21:08:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.921+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc010a1a0 con 0x7f7bc01033f0 2024-04-26T21:08:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.923+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7bb0004c00 con 0x7f7bc01033f0 2024-04-26T21:08:50.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.925+0000 7f7bb7e00640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.925+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7bb00cfb70 con 0x7f7bc01033f0 2024-04-26T21:08:50.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.925+0000 7f7bb6000640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.926+0000 7f7bb6000640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8006010 tx=0x7f7ba80095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:50.929+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7bb009c950 con 0x7f7bc01033f0 2024-04-26T21:08:50.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:50 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/623100838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:50 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/623100838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:08:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.175+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bc002dad0 con 0x7f7bc01033f0 2024-04-26T21:08:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.176+0000 7f7bb7e00640 1 -- 172.21.15.27:0/1414673990 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f7bb009c250 con 0x7f7bc01033f0 2024-04-26T21:08:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:51.177 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:08:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-04-26T21:08:51.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 msgr2=0x7f7ba40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8006010 tx=0x7f7ba80095f0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 msgr2=0x7f7bc019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000d770 tx=0x7f7bb000dc30 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 shutdown_connections 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7ba40a3bb0 0x7f7ba40a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc0105d20 0x7f7bc019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc0108650 0x7f7bc01a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 --2- 172.21.15.27:0/1414673990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bc01033f0 0x7f7bc019ad50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.178+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 >> 172.21.15.27:0/1414673990 conn(0x7f7bc00fcde0 msgr2=0x7f7bc0102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.179+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 shutdown_connections 2024-04-26T21:08:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:51.179+0000 7f7bc5600640 1 -- 172.21.15.27:0/1414673990 wait complete. 2024-04-26T21:08:51.937 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:51 smithi204 ceph-mon[32145]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:51.937 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/972468532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:51.937 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1414673990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:51 smithi027 ceph-mon[28085]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/972468532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:51.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1414673990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:51 smithi094 ceph-mon[31273]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/972468532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:08:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1414673990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:52.200 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:51 smithi204 podman[62108]: 2024-04-26 21:08:51.960498931 +0000 UTC m=+0.155731012 container create 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:08:52.200 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 podman[62108]: 2024-04-26 21:08:52.197616312 +0000 UTC m=+0.392848440 container init 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T21:08:52.502 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 podman[62108]: 2024-04-26 21:08:52.207895409 +0000 UTC m=+0.403127463 container start 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:08:52.502 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 ceph-mds[62127]: -- 172.21.15.204:0/2034675716 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed7777da40 con 0x55ed77751000 2024-04-26T21:08:52.502 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: starting mds.l at 2024-04-26T21:08:52.502 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 bash[62108]: 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d 2024-04-26T21:08:52.503 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:08:52 smithi204 systemd[1]: Started Ceph mds.l for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:08:53.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 -- 172.21.15.27:0/2614745999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff264108d90 msgr2=0x7ff2641091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 --2- 172.21.15.27:0/2614745999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff264108d90 0x7ff2641091b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402a410 comp rx=0 tx=0).stop 2024-04-26T21:08:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 -- 172.21.15.27:0/2614745999 shutdown_connections 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 --2- 172.21.15.27:0/2614745999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 0x7ff26410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 --2- 172.21.15.27:0/2614745999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff26410a940 0x7ff2641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 --2- 172.21.15.27:0/2614745999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff264108d90 0x7ff2641091b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 -- 172.21.15.27:0/2614745999 >> 172.21.15.27:0/2614745999 conn(0x7ff264077d30 msgr2=0x7ff26407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 -- 172.21.15.27:0/2614745999 shutdown_connections 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.426+0000 7ff268a00640 1 -- 172.21.15.27:0/2614745999 wait complete. 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 Processor -- start 2024-04-26T21:08:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 -- start start 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff264108d90 0x7ff26419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 0x7ff26419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff264114760 con 0x7ff26410a940 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641145e0 con 0x7ff264108d90 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.428+0000 7ff268a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641148e0 con 0x7ff264109f80 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37612/0 (socket says 172.21.15.27:37612) 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 -- 172.21.15.27:0/764972578 learned_addr learned my addr 172.21.15.27:0/764972578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263400640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff264108d90 0x7ff26419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff25be00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 0x7ff26419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 -- 172.21.15.27:0/764972578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 msgr2=0x7ff26419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 0x7ff26419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 -- 172.21.15.27:0/764972578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff264108d90 msgr2=0x7ff26419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff264108d90 0x7ff26419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.429+0000 7ff263e00640 1 -- 172.21.15.27:0/764972578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641a6b20 con 0x7ff26410a940 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff263e00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eb10 tx=0x7ff25c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004b20 con 0x7ff26410a940 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff25c004cc0 con 0x7ff26410a940 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a6e10 con 0x7ff26410a940 2024-04-26T21:08:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c010690 con 0x7ff26410a940 2024-04-26T21:08:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.430+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a72d0 con 0x7ff26410a940 2024-04-26T21:08:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.431+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264110270 con 0x7ff26410a940 2024-04-26T21:08:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.434+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff25c005020 con 0x7ff26410a940 2024-04-26T21:08:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.435+0000 7ff260c00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 0x7ff2480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.435+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff25c0d0650 con 0x7ff26410a940 2024-04-26T21:08:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.435+0000 7ff263400640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 0x7ff2480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.436+0000 7ff263400640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 0x7ff2480a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff2540095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.442+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff25c09d430 con 0x7ff26410a940 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6832/3519578339,v1:172.21.15.204:6833/3519578339] up:boot 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: fsmap 1 up:standby 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:53.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:53 smithi094 ceph-mon[31273]: Deploying daemon mds.k on smithi094 2024-04-26T21:08:53.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.693+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff26402dad0 con 0x7ff26410a940 2024-04-26T21:08:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.694+0000 7ff260c00640 1 -- 172.21.15.27:0/764972578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ff25c09cd30 con 0x7ff26410a940 2024-04-26T21:08:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:53.696 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:08:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-04-26T21:08:53.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 msgr2=0x7ff2480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 0x7ff2480a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff254006010 tx=0x7ff2540095f0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 msgr2=0x7ff2641a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eb10 tx=0x7ff25c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 shutdown_connections 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff2480a3c30 0x7ff2480a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264109f80 0x7ff26419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff26410a940 0x7ff2641a64b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 --2- 172.21.15.27:0/764972578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff264108d90 0x7ff26419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 >> 172.21.15.27:0/764972578 conn(0x7ff264077d30 msgr2=0x7ff26410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 shutdown_connections 2024-04-26T21:08:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:53.697+0000 7ff268a00640 1 -- 172.21.15.27:0/764972578 wait complete. 2024-04-26T21:08:53.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:53.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6832/3519578339,v1:172.21.15.204:6833/3519578339] up:boot 2024-04-26T21:08:53.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: fsmap 1 up:standby 2024-04-26T21:08:53.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:53.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:53 smithi027 ceph-mon[28085]: Deploying daemon mds.k on smithi094 2024-04-26T21:08:53.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:53.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6832/3519578339,v1:172.21.15.204:6833/3519578339] up:boot 2024-04-26T21:08:53.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: fsmap 1 up:standby 2024-04-26T21:08:53.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:53.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:53 smithi204 ceph-mon[32145]: Deploying daemon mds.k on smithi094 2024-04-26T21:08:54.672 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:54 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/764972578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:54.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:54 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/764972578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:54 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/764972578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:55.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:55.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:55 smithi027 ceph-mon[28085]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:55.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:55 smithi204 ceph-mon[32145]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:55.755 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:55 smithi094 ceph-mon[31273]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:55.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 -- 172.21.15.27:0/704778019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 msgr2=0x7fc53410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 --2- 172.21.15.27:0/704778019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53410a600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc520009900 tx=0x7fc52002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 -- 172.21.15.27:0/704778019 shutdown_connections 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 --2- 172.21.15.27:0/704778019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 0x7fc5341113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 --2- 172.21.15.27:0/704778019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53410a600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 --2- 172.21.15.27:0/704778019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 0x7fc5341093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 -- 172.21.15.27:0/704778019 >> 172.21.15.27:0/704778019 conn(0x7fc534077f40 msgr2=0x7fc53407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 -- 172.21.15.27:0/704778019 shutdown_connections 2024-04-26T21:08:55.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.935+0000 7fc53c400640 1 -- 172.21.15.27:0/704778019 wait complete. 2024-04-26T21:08:55.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 Processor -- start 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 -- start start 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 0x7fc53419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 0x7fc5341a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc534114980 con 0x7fc53410a180 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc534114800 con 0x7fc53410ab40 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.936+0000 7fc53c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc534114b00 con 0x7fc534108f90 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37638/0 (socket says 172.21.15.27:37638) 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 -- 172.21.15.27:0/3936842175 learned_addr learned my addr 172.21.15.27:0/3936842175 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc53a400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 0x7fc5341a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539a00640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 0x7fc53419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 -- 172.21.15.27:0/3936842175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 msgr2=0x7fc53419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 0x7fc53419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 -- 172.21.15.27:0/3936842175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 msgr2=0x7fc5341a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 0x7fc5341a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.937+0000 7fc539000640 1 -- 172.21.15.27:0/3936842175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341a6d00 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc539000640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc520009a20 tx=0x7fc5200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52003f030 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc520003a80 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341a6f90 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520035070 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.938+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341a73f0 con 0x7fc53410a180 2024-04-26T21:08:55.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.939+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc534110300 con 0x7fc53410a180 2024-04-26T21:08:55.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.941+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc520042030 con 0x7fc53410a180 2024-04-26T21:08:55.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.943+0000 7fc532000640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.943+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc5200f0640 con 0x7fc53410a180 2024-04-26T21:08:55.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.943+0000 7fc539a00640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.944+0000 7fc539a00640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc528005fd0 tx=0x7fc528005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:55.948+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc5200bd420 con 0x7fc53410a180 2024-04-26T21:08:56.027 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:55 smithi094 podman[61232]: 2024-04-26 21:08:55.815925024 +0000 UTC m=+0.147642795 container create 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:08:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.196+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc534063560 con 0x7fc53410a180 2024-04-26T21:08:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.197+0000 7fc532000640 1 -- 172.21.15.27:0/3936842175 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc5200bcd20 con 0x7fc53410a180 2024-04-26T21:08:56.198 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:56.198 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:08:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 msgr2=0x7fc51c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc528005fd0 tx=0x7fc528005850 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 msgr2=0x7fc53419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc520009a20 tx=0x7fc5200040c0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 shutdown_connections 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc51c0a3bb0 0x7fc51c0a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53410ab40 0x7fc5341a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc53410a180 0x7fc53419f620 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 --2- 172.21.15.27:0/3936842175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc534108f90 0x7fc53419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.199+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 >> 172.21.15.27:0/3936842175 conn(0x7fc534077f40 msgr2=0x7fc53410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.200+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 shutdown_connections 2024-04-26T21:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:56.200+0000 7fc53c400640 1 -- 172.21.15.27:0/3936842175 wait complete. 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3936842175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 podman[61232]: 2024-04-26 21:08:56.028200058 +0000 UTC m=+0.359917870 container init 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 podman[61232]: 2024-04-26 21:08:56.037711656 +0000 UTC m=+0.369429407 container start 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:08:56.473 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mds[61255]: -- 172.21.15.94:0/3608186140 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c62a93a40 con 0x562c62a66c00 2024-04-26T21:08:56.474 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: starting mds.k at 2024-04-26T21:08:56.474 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 bash[61232]: 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 2024-04-26T21:08:56.474 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:08:56 smithi094 systemd[1]: Started Ceph mds.k for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3936842175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:56 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3936842175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:08:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:56.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:56 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:08:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:08:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:56 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:57.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:57 smithi027 ceph-mon[28085]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:57.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:57 smithi027 ceph-mon[28085]: Deploying daemon mds.j on smithi027 2024-04-26T21:08:57.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:57 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:boot 2024-04-26T21:08:57.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:57 smithi027 ceph-mon[28085]: fsmap 2 up:standby 2024-04-26T21:08:57.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:57 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:08:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:57 smithi204 ceph-mon[32145]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:57 smithi204 ceph-mon[32145]: Deploying daemon mds.j on smithi027 2024-04-26T21:08:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:57 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:boot 2024-04-26T21:08:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:57 smithi204 ceph-mon[32145]: fsmap 2 up:standby 2024-04-26T21:08:57.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:57 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:08:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:57 smithi094 ceph-mon[31273]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:57 smithi094 ceph-mon[31273]: Deploying daemon mds.j on smithi027 2024-04-26T21:08:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:57 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:boot 2024-04-26T21:08:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:57 smithi094 ceph-mon[31273]: fsmap 2 up:standby 2024-04-26T21:08:57.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:57 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:08:58.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1477425356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac10a800 msgr2=0x7f1fac10cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1477425356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac10a800 0x7f1fac10cca0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009ca0 tx=0x7f1fa402a940 comp rx=0 tx=0).stop 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1477425356 shutdown_connections 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1477425356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac075330 0x7f1fac075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1477425356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac10a800 0x7f1fac10cca0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1477425356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 0x7f1fac074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1477425356 >> 172.21.15.27:0/1477425356 conn(0x7f1fac0fdee0 msgr2=0x7f1fac100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1477425356 shutdown_connections 2024-04-26T21:08:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.439+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1477425356 wait complete. 2024-04-26T21:08:58.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.440+0000 7f1fb3c00640 1 Processor -- start 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 -- start start 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 0x7f1fac19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 0x7f1fac1a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac110580 con 0x7f1fac075330 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac110400 con 0x7f1fac076910 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb3c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac110700 con 0x7f1fac10a800 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb1200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37666/0 (socket says 172.21.15.27:37666) 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.441+0000 7f1fb1200640 1 -- 172.21.15.27:0/1825907448 learned_addr learned my addr 172.21.15.27:0/1825907448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fabe00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 0x7f1fac19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 0x7f1fac1a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 -- 172.21.15.27:0/1825907448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 msgr2=0x7f1fac1a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 0x7f1fac1a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 -- 172.21.15.27:0/1825907448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 msgr2=0x7f1fac19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 0x7f1fac19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 -- 172.21.15.27:0/1825907448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac1a2780 con 0x7f1fac075330 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 0x7f1fac1a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fb1200640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c920 tx=0x7f1fa000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.442+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007c70 con 0x7f1fac075330 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.443+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa0007e10 con 0x7f1fac075330 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.443+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac1a2a70 con 0x7f1fac075330 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.443+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0018710 con 0x7f1fac075330 2024-04-26T21:08:58.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.443+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac1a2fc0 con 0x7f1fac075330 2024-04-26T21:08:58.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.444+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fac1095c0 con 0x7f1fac075330 2024-04-26T21:08:58.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.446+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1fa000f450 con 0x7f1fac075330 2024-04-26T21:08:58.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.448+0000 7f1fa9600640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 0x7f1f940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:58.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.449+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1fa00d0270 con 0x7f1fac075330 2024-04-26T21:08:58.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.449+0000 7f1fabe00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 0x7f1f940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:58.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.449+0000 7f1fabe00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 0x7f1f940a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f98005e10 tx=0x7f1f98005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.453+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1fa0014030 con 0x7f1fac075330 2024-04-26T21:08:58.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.703+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fac02dad0 con 0x7f1fac075330 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.705+0000 7f1fa9600640 1 -- 172.21.15.27:0/1825907448 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f1fac02dad0 con 0x7f1fac075330 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:08:58.706 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:08:58.707 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.707+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 msgr2=0x7f1f940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.707+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 0x7f1f940a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f98005e10 tx=0x7f1f98005da0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.707+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 msgr2=0x7f1fac19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.707+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c920 tx=0x7f1fa000cde0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 shutdown_connections 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1f940a3bb0 0x7f1f940a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fac10a800 0x7f1fac1a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fac076910 0x7f1fac19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 --2- 172.21.15.27:0/1825907448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fac075330 0x7f1fac19ab60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:58.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 >> 172.21.15.27:0/1825907448 conn(0x7f1fac0fdee0 msgr2=0x7f1fac106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:58.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 shutdown_connections 2024-04-26T21:08:58.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:08:58.708+0000 7f1fb3c00640 1 -- 172.21.15.27:0/1825907448 wait complete. 2024-04-26T21:08:59.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:59 smithi027 ceph-mon[28085]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:59.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:08:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1825907448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:59 smithi204 ceph-mon[32145]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:59.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:08:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1825907448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:59.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:59 smithi094 ceph-mon[31273]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:08:59.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:08:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1825907448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:08:59.919 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:08:59 smithi027 systemd[1]: Starting Ceph mds.j for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:09:00.222 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:08:59 smithi027 podman[68567]: 2024-04-26 21:08:59.821987388 +0000 UTC m=+0.037342250 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:00.222 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 podman[68567]: 2024-04-26 21:09:00.0230996 +0000 UTC m=+0.238454502 container create 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:09:00.730 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:00.737 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 podman[68567]: 2024-04-26 21:09:00.317994532 +0000 UTC m=+0.533349463 container init 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:09:00.737 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 podman[68567]: 2024-04-26 21:09:00.328837219 +0000 UTC m=+0.544192105 container start 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:09:00.738 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 ceph-mds[68586]: -- 172.21.15.27:0/2796730123 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564796f77a40 con 0x564796f4ac00 2024-04-26T21:09:00.738 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j[68582]: starting mds.j at 2024-04-26T21:09:00.738 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 bash[68567]: 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 2024-04-26T21:09:00.738 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:09:00 smithi027 systemd[1]: Started Ceph mds.j for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 -- 172.21.15.27:0/1519183958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 msgr2=0x7f26101091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 --2- 172.21.15.27:0/1519183958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f26101091b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2600009900 tx=0x7f260002a400 comp rx=0 tx=0).stop 2024-04-26T21:09:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 -- 172.21.15.27:0/1519183958 shutdown_connections 2024-04-26T21:09:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 --2- 172.21.15.27:0/1519183958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2610109f80 0x7f261010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 --2- 172.21.15.27:0/1519183958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f261010a940 0x7f26101111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 --2- 172.21.15.27:0/1519183958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f26101091b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.930+0000 7f2616200640 1 -- 172.21.15.27:0/1519183958 >> 172.21.15.27:0/1519183958 conn(0x7f2610077d30 msgr2=0x7f261007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.931+0000 7f2616200640 1 -- 172.21.15.27:0/1519183958 shutdown_connections 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.931+0000 7f2616200640 1 -- 172.21.15.27:0/1519183958 wait complete. 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 Processor -- start 2024-04-26T21:09:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 -- start start 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2610109f80 0x7f261019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 0x7f26101a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2610114790 con 0x7f2610108d90 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2610114610 con 0x7f2610109f80 2024-04-26T21:09:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2616200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2610114910 con 0x7f261010a940 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2614e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2614e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40002/0 (socket says 172.21.15.27:40002) 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.932+0000 7f2614e00640 1 -- 172.21.15.27:0/3225654894 learned_addr learned my addr 172.21.15.27:0/3225654894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2607e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2610109f80 0x7f261019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 -- 172.21.15.27:0/3225654894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 msgr2=0x7f26101a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2615800640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 0x7f26101a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 0x7f26101a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 -- 172.21.15.27:0/3225654894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2610109f80 msgr2=0x7f261019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2610109f80 0x7f261019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 -- 172.21.15.27:0/3225654894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101a69f0 con 0x7f2610108d90 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2615800640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 0x7f26101a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.933+0000 7f2614e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2600031460 tx=0x7f2600003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.934+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260003f070 con 0x7f2610108d90 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.934+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101a6c80 con 0x7f2610108d90 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.934+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2600031da0 con 0x7f2610108d90 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.934+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600035030 con 0x7f2610108d90 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.934+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a71c0 con 0x7f2610108d90 2024-04-26T21:09:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.935+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610110130 con 0x7f2610108d90 2024-04-26T21:09:00.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.937+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2600042070 con 0x7f2610108d90 2024-04-26T21:09:00.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.938+0000 7f260e000640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 0x7f25f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.939+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f26000f0a20 con 0x7f2610108d90 2024-04-26T21:09:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.939+0000 7f2607e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 0x7f25f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.940+0000 7f2607e00640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 0x7f25f40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005e10 tx=0x7f25f8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:00.943+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26000bd800 con 0x7f2610108d90 2024-04-26T21:09:01.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.250+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2610063530 con 0x7f2610108d90 2024-04-26T21:09:01.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.251+0000 7f260e000640 1 -- 172.21.15.27:0/3225654894 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f26000bd100 con 0x7f2610108d90 2024-04-26T21:09:01.252 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:01.252 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:09:01.252 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-04-26T21:09:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 msgr2=0x7f25f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 0x7f25f40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005e10 tx=0x7f25f8005da0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 msgr2=0x7f261019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2600031460 tx=0x7f2600003d50 comp rx=0 tx=0).stop 2024-04-26T21:09:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 shutdown_connections 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25f40a3bb0 0x7f25f40a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f261010a940 0x7f26101a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2610109f80 0x7f261019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 --2- 172.21.15.27:0/3225654894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2610108d90 0x7f261019edd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 >> 172.21.15.27:0/3225654894 conn(0x7f2610077d30 msgr2=0x7f261010d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 shutdown_connections 2024-04-26T21:09:01.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:01.254+0000 7f2616200640 1 -- 172.21.15.27:0/3225654894 wait complete. 2024-04-26T21:09:01.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6834/2990870611,v1:172.21.15.27:6835/2990870611] up:boot 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: fsmap 3 up:standby 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: Deploying daemon mds.d on smithi027 2024-04-26T21:09:01.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:01 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3225654894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6834/2990870611,v1:172.21.15.27:6835/2990870611] up:boot 2024-04-26T21:09:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: fsmap 3 up:standby 2024-04-26T21:09:01.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: Deploying daemon mds.d on smithi027 2024-04-26T21:09:01.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:01 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3225654894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6834/2990870611,v1:172.21.15.27:6835/2990870611] up:boot 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: fsmap 3 up:standby 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:01.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:01.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:01.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: Deploying daemon mds.d on smithi027 2024-04-26T21:09:01.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:01 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3225654894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:03.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 -- 172.21.15.27:0/301460853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 msgr2=0x7fc3ac103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/301460853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac103ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc398009900 tx=0x7fc39802a410 comp rx=0 tx=0).stop 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 -- 172.21.15.27:0/301460853 shutdown_connections 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/301460853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac104530 0x7fc3ac10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/301460853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac103ff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/301460853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac102980 0x7fc3ac102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 -- 172.21.15.27:0/301460853 >> 172.21.15.27:0/301460853 conn(0x7fc3ac0fdf10 msgr2=0x7fc3ac100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 -- 172.21.15.27:0/301460853 shutdown_connections 2024-04-26T21:09:03.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.501+0000 7fc3b2a00640 1 -- 172.21.15.27:0/301460853 wait complete. 2024-04-26T21:09:03.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 Processor -- start 2024-04-26T21:09:03.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 -- start start 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac102980 0x7fc3ac19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac104530 0x7fc3ac1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac10e3b0 con 0x7fc3ac103b70 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac10e230 con 0x7fc3ac104530 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.503+0000 7fc3b2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac10e530 con 0x7fc3ac102980 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40022/0 (socket says 172.21.15.27:40022) 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 -- 172.21.15.27:0/3111798100 learned_addr learned my addr 172.21.15.27:0/3111798100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3abe00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac102980 0x7fc3ac19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3b0a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac104530 0x7fc3ac1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 -- 172.21.15.27:0/3111798100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac102980 msgr2=0x7fc3ac19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac102980 0x7fc3ac19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 -- 172.21.15.27:0/3111798100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac104530 msgr2=0x7fc3ac1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac104530 0x7fc3ac1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.504+0000 7fc3ab400640 1 -- 172.21.15.27:0/3111798100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a2900 con 0x7fc3ac103b70 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3ab400640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc398031460 tx=0x7fc398003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39803f070 con 0x7fc3ac103b70 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc398031ba0 con 0x7fc3ac103b70 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398035030 con 0x7fc3ac103b70 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a2b90 con 0x7fc3ac103b70 2024-04-26T21:09:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.505+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a3030 con 0x7fc3ac103b70 2024-04-26T21:09:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.507+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac1a3200 con 0x7fc3ac103b70 2024-04-26T21:09:03.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.509+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc398042070 con 0x7fc3ac103b70 2024-04-26T21:09:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.510+0000 7fc3a8c00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.511+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc3980f0fa0 con 0x7fc3ac103b70 2024-04-26T21:09:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.511+0000 7fc3abe00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.511+0000 7fc3abe00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 0x7fc3900a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac19bb90 tx=0x7fc39c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.519+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc3980bdd80 con 0x7fc3ac103b70 2024-04-26T21:09:03.731 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:03 smithi027 ceph-mon[28085]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:03.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:03 smithi204 ceph-mon[32145]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:03.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.772+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3ac1a3540 con 0x7fc3ac103b70 2024-04-26T21:09:03.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.772+0000 7fc3a8c00640 1 -- 172.21.15.27:0/3111798100 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fc3980bd680 con 0x7fc3ac103b70 2024-04-26T21:09:03.773 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:03.774 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:09:03.774 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-04-26T21:09:03.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:03 smithi094 ceph-mon[31273]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:03.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 msgr2=0x7fc3900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 0x7fc3900a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac19bb90 tx=0x7fc39c005f90 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 msgr2=0x7fc3ac19b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc398031460 tx=0x7fc398003c10 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 shutdown_connections 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc3900a3c30 0x7fc3900a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3ac104530 0x7fc3ac1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3ac103b70 0x7fc3ac19b200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 --2- 172.21.15.27:0/3111798100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3ac102980 0x7fc3ac19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 >> 172.21.15.27:0/3111798100 conn(0x7fc3ac0fdf10 msgr2=0x7fc3ac074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 shutdown_connections 2024-04-26T21:09:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:03.775+0000 7fc3b2a00640 1 -- 172.21.15.27:0/3111798100 wait complete. 2024-04-26T21:09:04.376 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 podman[68953]: 2024-04-26 21:09:04.144319577 +0000 UTC m=+0.175685701 container create fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T21:09:04.376 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 podman[68953]: 2024-04-26 21:09:04.374099781 +0000 UTC m=+0.405465949 container init fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T21:09:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:04 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3111798100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:04.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 podman[68953]: 2024-04-26 21:09:04.385203018 +0000 UTC m=+0.416569112 container start fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T21:09:04.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 ceph-mds[68972]: -- 172.21.15.27:0/1923301766 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594f5561a40 con 0x5594f5534800 2024-04-26T21:09:04.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d[68968]: starting mds.d at 2024-04-26T21:09:04.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 bash[68953]: fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 2024-04-26T21:09:04.736 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:09:04 smithi027 systemd[1]: Started Ceph mds.d for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:04.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:04 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3111798100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:04 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3111798100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6836/1830505768,v1:172.21.15.27:6837/1830505768] up:boot 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: fsmap 4 up:standby 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:05.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:05 smithi094 ceph-mon[31273]: Deploying daemon mds.e on smithi094 2024-04-26T21:09:05.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6836/1830505768,v1:172.21.15.27:6837/1830505768] up:boot 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: fsmap 4 up:standby 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:05.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:05.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:05.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:05.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:05 smithi027 ceph-mon[28085]: Deploying daemon mds.e on smithi094 2024-04-26T21:09:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6836/1830505768,v1:172.21.15.27:6837/1830505768] up:boot 2024-04-26T21:09:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: fsmap 4 up:standby 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:06.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:05 smithi204 ceph-mon[32145]: Deploying daemon mds.e on smithi094 2024-04-26T21:09:06.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 -- 172.21.15.27:0/4292721394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98103b40 msgr2=0x7fba98103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:06.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 --2- 172.21.15.27:0/4292721394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98103b40 0x7fba98103fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba80009900 tx=0x7fba8002a410 comp rx=0 tx=0).stop 2024-04-26T21:09:06.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 -- 172.21.15.27:0/4292721394 shutdown_connections 2024-04-26T21:09:06.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 --2- 172.21.15.27:0/4292721394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 0x7fba9810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 --2- 172.21.15.27:0/4292721394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98103b40 0x7fba98103fc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 --2- 172.21.15.27:0/4292721394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98102950 0x7fba98102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 -- 172.21.15.27:0/4292721394 >> 172.21.15.27:0/4292721394 conn(0x7fba980fdee0 msgr2=0x7fba98100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 -- 172.21.15.27:0/4292721394 shutdown_connections 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.026+0000 7fba9f800640 1 -- 172.21.15.27:0/4292721394 wait complete. 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.027+0000 7fba9f800640 1 Processor -- start 2024-04-26T21:09:06.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.027+0000 7fba9f800640 1 -- start start 2024-04-26T21:09:06.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:06.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98103b40 0x7fba9819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 0x7fba981a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba9810e380 con 0x7fba98102950 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba9810e200 con 0x7fba98104500 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9f800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba9810e500 con 0x7fba98103b40 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40058/0 (socket says 172.21.15.27:40058) 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.028+0000 7fba9ce00640 1 -- 172.21.15.27:0/4261597308 learned_addr learned my addr 172.21.15.27:0/4261597308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9d800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 0x7fba981a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:06.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba97e00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98103b40 0x7fba9819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:06.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 -- 172.21.15.27:0/4261597308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98103b40 msgr2=0x7fba9819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:06.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98103b40 0x7fba9819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 -- 172.21.15.27:0/4261597308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 msgr2=0x7fba981a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:06.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 0x7fba981a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 -- 172.21.15.27:0/4261597308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba981a28f0 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.029+0000 7fba9ce00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba8800eaa0 tx=0x7fba8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.030+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88016070 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.030+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8800ce10 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.030+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba981a2be0 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.030+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88011470 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.030+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba981a30b0 con 0x7fba98102950 2024-04-26T21:09:06.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.031+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba981a3280 con 0x7fba98102950 2024-04-26T21:09:06.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.033+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fba88019070 con 0x7fba98102950 2024-04-26T21:09:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.034+0000 7fba95600640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 0x7fba7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.035+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fba880cfff0 con 0x7fba98102950 2024-04-26T21:09:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.035+0000 7fba97e00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 0x7fba7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:06.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.036+0000 7fba97e00640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 0x7fba7c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba80005ab0 tx=0x7fba80005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:06.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.040+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fba8809c780 con 0x7fba98102950 2024-04-26T21:09:06.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.291+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba981a3450 con 0x7fba98102950 2024-04-26T21:09:06.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.292+0000 7fba95600640 1 -- 172.21.15.27:0/4261597308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fba880a0a80 con 0x7fba98102950 2024-04-26T21:09:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:06.294 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:09:06.294 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2024-04-26T21:09:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 msgr2=0x7fba7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:06.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 0x7fba7c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba80005ab0 tx=0x7fba80005a20 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 msgr2=0x7fba9819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba8800eaa0 tx=0x7fba8800ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 shutdown_connections 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba7c0a3bb0 0x7fba7c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba98103b40 0x7fba9819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba98104500 0x7fba981a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 --2- 172.21.15.27:0/4261597308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba98102950 0x7fba9819ad00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 >> 172.21.15.27:0/4261597308 conn(0x7fba980fdee0 msgr2=0x7fba98074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 shutdown_connections 2024-04-26T21:09:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:06.295+0000 7fba9f800640 1 -- 172.21.15.27:0/4261597308 wait complete. 2024-04-26T21:09:06.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:06 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4261597308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:06.800 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:06 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4261597308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:07.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:06 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4261597308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:07.814 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:07 smithi094 ceph-mon[31273]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:07 smithi027 ceph-mon[28085]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:07 smithi204 ceph-mon[32145]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:08.166 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:07 smithi094 systemd[1]: Starting Ceph mds.e for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:09:08.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:08.461 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 podman[61538]: 2024-04-26 21:09:08.070138604 +0000 UTC m=+0.030940889 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:08.461 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 podman[61538]: 2024-04-26 21:09:08.188766169 +0000 UTC m=+0.149568491 container create f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-26T21:09:08.461 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 podman[61538]: 2024-04-26 21:09:08.459619634 +0000 UTC m=+0.420421962 container init f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T21:09:08.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 -- 172.21.15.27:0/97027264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 msgr2=0x7fa2e010cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 --2- 172.21.15.27:0/97027264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e010cdd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009ca0 tx=0x7fa2dc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 -- 172.21.15.27:0/97027264 shutdown_connections 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 --2- 172.21.15.27:0/97027264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e010cdd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 --2- 172.21.15.27:0/97027264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0105b80 0x7fa2e0106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 --2- 172.21.15.27:0/97027264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0104990 0x7fa2e0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 -- 172.21.15.27:0/97027264 >> 172.21.15.27:0/97027264 conn(0x7fa2e00fff00 msgr2=0x7fa2e0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 -- 172.21.15.27:0/97027264 shutdown_connections 2024-04-26T21:09:08.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.543+0000 7fa2e7200640 1 -- 172.21.15.27:0/97027264 wait complete. 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.544+0000 7fa2e7200640 1 Processor -- start 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 -- start start 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 0x7fa2e019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e01a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01103c0 con 0x7fa2e0106540 2024-04-26T21:09:08.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0110240 con 0x7fa2e0104990 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0110540 con 0x7fa2e0105b80 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e01a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:47102/0 (socket says 172.21.15.27:47102) 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.545+0000 7fa2e5e00640 1 -- 172.21.15.27:0/640800402 learned_addr learned my addr 172.21.15.27:0/640800402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e5400640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 0x7fa2e019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 -- 172.21.15.27:0/640800402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 msgr2=0x7fa2e019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 0x7fa2e019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 -- 172.21.15.27:0/640800402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 msgr2=0x7fa2e019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 -- 172.21.15.27:0/640800402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e01a26a0 con 0x7fa2e0106540 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e5e00640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e5400640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 0x7fa2e019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.546+0000 7fa2e6800640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e01a2030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc031040 tx=0x7fa2dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.547+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc032ed0 con 0x7fa2e0106540 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.547+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2dc0044d0 con 0x7fa2e0106540 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.547+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc030c00 con 0x7fa2e0106540 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.547+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e01a2930 con 0x7fa2e0106540 2024-04-26T21:09:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.547+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e01a2e30 con 0x7fa2e0106540 2024-04-26T21:09:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.548+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e010bd80 con 0x7fa2e0106540 2024-04-26T21:09:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.550+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa2dc004040 con 0x7fa2e0106540 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.552+0000 7fa2daa00640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 0x7fa2c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.552+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa2dc0f1450 con 0x7fa2e0106540 2024-04-26T21:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.552+0000 7fa2e5e00640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 0x7fa2c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.553+0000 7fa2e5e00640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 0x7fa2c80a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fd0 tx=0x7fa2d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.556+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2dc0be230 con 0x7fa2e0106540 2024-04-26T21:09:08.776 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 podman[61538]: 2024-04-26 21:09:08.470112607 +0000 UTC m=+0.430914897 container start f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:09:08.776 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 ceph-mds[61557]: -- 172.21.15.94:0/2908279602 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d4230ba40 con 0x556d422df000 2024-04-26T21:09:08.776 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: starting mds.e at 2024-04-26T21:09:08.777 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 bash[61538]: f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b 2024-04-26T21:09:08.777 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:09:08 smithi094 systemd[1]: Started Ceph mds.e for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.810+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2e002dad0 con 0x7fa2e0106540 2024-04-26T21:09:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.812+0000 7fa2daa00640 1 -- 172.21.15.27:0/640800402 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fa2dc0bdb30 con 0x7fa2e0106540 2024-04-26T21:09:08.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:08.813 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:09:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.814+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 msgr2=0x7fa2c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 0x7fa2c80a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fd0 tx=0x7fa2d400b410 comp rx=0 tx=0).stop 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 msgr2=0x7fa2e01a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e01a2030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc031040 tx=0x7fa2dc003ba0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 shutdown_connections 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa2c80a3bb0 0x7fa2c80a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2e0105b80 0x7fa2e019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e0106540 0x7fa2e01a2030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 --2- 172.21.15.27:0/640800402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2e0104990 0x7fa2e019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 >> 172.21.15.27:0/640800402 conn(0x7fa2e00fff00 msgr2=0x7fa2e0108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 shutdown_connections 2024-04-26T21:09:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:08.815+0000 7fa2e7200640 1 -- 172.21.15.27:0/640800402 wait complete. 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:boot 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: fsmap 5 up:standby 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:09.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: Deploying daemon mds.f on smithi204 2024-04-26T21:09:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:09 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/640800402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:09.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:boot 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: fsmap 5 up:standby 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: Deploying daemon mds.f on smithi204 2024-04-26T21:09:09.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/640800402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:boot 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: fsmap 5 up:standby 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: Deploying daemon mds.f on smithi204 2024-04-26T21:09:10.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:09 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/640800402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:10.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 -- 172.21.15.27:0/3451783044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c075330 msgr2=0x7f312c075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 --2- 172.21.15.27:0/3451783044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c075330 0x7f312c075790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3114009900 tx=0x7f311402a480 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 -- 172.21.15.27:0/3451783044 shutdown_connections 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 --2- 172.21.15.27:0/3451783044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 --2- 172.21.15.27:0/3451783044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c075330 0x7f312c075790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 --2- 172.21.15.27:0/3451783044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c076910 0x7f312c074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.058+0000 7f312b200640 1 -- 172.21.15.27:0/3451783044 >> 172.21.15.27:0/3451783044 conn(0x7f312c0fdd40 msgr2=0x7f312c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.059+0000 7f312b200640 1 -- 172.21.15.27:0/3451783044 shutdown_connections 2024-04-26T21:09:11.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.059+0000 7f312b200640 1 -- 172.21.15.27:0/3451783044 wait complete. 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 Processor -- start 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 -- start start 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c075330 0x7f312c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f312c110370 con 0x7f312c076910 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1101f0 con 0x7f312c10a640 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f312b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1104f0 con 0x7f312c075330 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f3123e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f3123e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41452/0 (socket says 172.21.15.27:41452) 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.060+0000 7f3123e00640 1 -- 172.21.15.27:0/1235534582 learned_addr learned my addr 172.21.15.27:0/1235534582 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3129e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c075330 0x7f312c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f312a800640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 -- 172.21.15.27:0/1235534582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c075330 msgr2=0x7f312c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c075330 0x7f312c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 -- 172.21.15.27:0/1235534582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 msgr2=0x7f312c1a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c1a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 -- 172.21.15.27:0/1235534582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1a2620 con 0x7f312c076910 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f312a800640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c1a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.061+0000 7f3123e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3114009230 tx=0x7f31140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.062+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311403f030 con 0x7f312c076910 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.062+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3114003c90 con 0x7f312c076910 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.062+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114035070 con 0x7f312c076910 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.062+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1a28b0 con 0x7f312c076910 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.062+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1a2d50 con 0x7f312c076910 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.063+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c109400 con 0x7f312c076910 2024-04-26T21:09:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.065+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3114042030 con 0x7f312c076910 2024-04-26T21:09:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.067+0000 7f3122a00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 0x7f31100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.068+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f31140f0640 con 0x7f312c076910 2024-04-26T21:09:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.068+0000 7f3129e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 0x7f31100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.069+0000 7f3129e00640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 0x7f31100a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f311c00abe0 tx=0x7f311c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.072+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31140bd420 con 0x7f312c076910 2024-04-26T21:09:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.320+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f312c02dad0 con 0x7f312c076910 2024-04-26T21:09:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.322+0000 7f3122a00640 1 -- 172.21.15.27:0/1235534582 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f31140bcd20 con 0x7f312c076910 2024-04-26T21:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:09:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-04-26T21:09:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 msgr2=0x7f31100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 0x7f31100a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f311c00abe0 tx=0x7f311c00a2d0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 msgr2=0x7f312c19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3114009230 tx=0x7f31140042f0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 shutdown_connections 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f31100a3c30 0x7f31100a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c10a640 0x7f312c1a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f312c076910 0x7f312c19af40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 --2- 172.21.15.27:0/1235534582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f312c075330 0x7f312c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.324+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 >> 172.21.15.27:0/1235534582 conn(0x7f312c0fdd40 msgr2=0x7f312c1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.325+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 shutdown_connections 2024-04-26T21:09:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:11.325+0000 7f312b200640 1 -- 172.21.15.27:0/1235534582 wait complete. 2024-04-26T21:09:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:11 smithi027 ceph-mon[28085]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:11.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:11 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1235534582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:11.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:11 smithi204 ceph-mon[32145]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:11.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:11 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1235534582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:11.971 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:11 smithi204 systemd[1]: Starting Ceph mds.f for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:09:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:11 smithi094 ceph-mon[31273]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:11 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1235534582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:12.253 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:11 smithi204 podman[62412]: 2024-04-26 21:09:11.875437107 +0000 UTC m=+0.036641181 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:12.253 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:11 smithi204 podman[62412]: 2024-04-26 21:09:11.979646756 +0000 UTC m=+0.140850871 container create 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T21:09:12.253 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 podman[62412]: 2024-04-26 21:09:12.208256189 +0000 UTC m=+0.369460307 container init 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:09:12.253 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 podman[62412]: 2024-04-26 21:09:12.217653548 +0000 UTC m=+0.378857640 container start 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:09:12.752 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 ceph-mds[62431]: -- 172.21.15.204:0/2915599281 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a728bda40 con 0x558a72890c00 2024-04-26T21:09:12.752 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: starting mds.f at 2024-04-26T21:09:12.752 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 bash[62412]: 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d 2024-04-26T21:09:12.752 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:09:12 smithi204 systemd[1]: Started Ceph mds.f for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:13.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:13.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 -- 172.21.15.27:0/3163264493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 msgr2=0x7f7f8410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 --2- 172.21.15.27:0/3163264493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f8410cdc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009900 tx=0x7f7f7c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 -- 172.21.15.27:0/3163264493 shutdown_connections 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 --2- 172.21.15.27:0/3163264493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84105b70 0x7f7f84105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 --2- 172.21.15.27:0/3163264493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f8410cdc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 --2- 172.21.15.27:0/3163264493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84104980 0x7f7f84104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 -- 172.21.15.27:0/3163264493 >> 172.21.15.27:0/3163264493 conn(0x7f7f840fff10 msgr2=0x7f7f84102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 -- 172.21.15.27:0/3163264493 shutdown_connections 2024-04-26T21:09:13.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.566+0000 7f7f83200640 1 -- 172.21.15.27:0/3163264493 wait complete. 2024-04-26T21:09:13.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.567+0000 7f7f83200640 1 Processor -- start 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.567+0000 7f7f83200640 1 -- start start 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 0x7f7f8419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84105b70 0x7f7f8419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841103b0 con 0x7f7f84106530 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84110230 con 0x7f7f84105b70 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f83200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84110530 con 0x7f7f84104980 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f82800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f81e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 0x7f7f8419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f81e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 0x7f7f8419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:59386/0 (socket says 172.21.15.27:59386) 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f82800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41468/0 (socket says 172.21.15.27:41468) 2024-04-26T21:09:13.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f81e00640 1 -- 172.21.15.27:0/808593847 learned_addr learned my addr 172.21.15.27:0/808593847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f81400640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84105b70 0x7f7f8419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f82800640 1 -- 172.21.15.27:0/808593847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 msgr2=0x7f7f8419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.568+0000 7f7f82800640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 0x7f7f8419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f82800640 1 -- 172.21.15.27:0/808593847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84105b70 msgr2=0x7f7f8419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f82800640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84105b70 0x7f7f8419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f82800640 1 -- 172.21.15.27:0/808593847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841a2670 con 0x7f7f84106530 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f82800640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02a910 tx=0x7f7f7c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c03f070 con 0x7f7f84106530 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f7c031b90 con 0x7f7f84106530 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a2900 con 0x7f7f84106530 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c035030 con 0x7f7f84106530 2024-04-26T21:09:13.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.569+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a2dc0 con 0x7f7f84106530 2024-04-26T21:09:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.571+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8410bca0 con 0x7f7f84106530 2024-04-26T21:09:13.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.573+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7f7c042070 con 0x7f7f84106530 2024-04-26T21:09:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.574+0000 7f7f7aa00640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 0x7f7f680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.575+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7f7c0f11b0 con 0x7f7f84106530 2024-04-26T21:09:13.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.575+0000 7f7f81e00640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 0x7f7f680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.575+0000 7f7f81e00640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 0x7f7f680a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009fa0 tx=0x7f7f740096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.577+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f7c0bdf90 con 0x7f7f84106530 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: Deploying daemon mds.c on smithi204 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6834/1241203213,v1:172.21.15.204:6835/1241203213] up:boot 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: fsmap 6 up:standby 2024-04-26T21:09:13.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:09:13.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: Deploying daemon mds.c on smithi204 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6834/1241203213,v1:172.21.15.204:6835/1241203213] up:boot 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: fsmap 6 up:standby 2024-04-26T21:09:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: Deploying daemon mds.c on smithi204 2024-04-26T21:09:13.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6834/1241203213,v1:172.21.15.204:6835/1241203213] up:boot 2024-04-26T21:09:13.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: fsmap 6 up:standby 2024-04-26T21:09:13.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.824+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f8402dad0 con 0x7f7f84106530 2024-04-26T21:09:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.825+0000 7f7f7aa00640 1 -- 172.21.15.27:0/808593847 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f7f7c0a6e20 con 0x7f7f84106530 2024-04-26T21:09:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:13.831 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:09:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 msgr2=0x7f7f680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 0x7f7f680a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009fa0 tx=0x7f7f740096d0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 msgr2=0x7f7f841a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02a910 tx=0x7f7f7c003b90 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 shutdown_connections 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7f680a3bb0 0x7f7f680a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f84106530 0x7f7f841a2000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f84105b70 0x7f7f8419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 --2- 172.21.15.27:0/808593847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f84104980 0x7f7f8419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 >> 172.21.15.27:0/808593847 conn(0x7f7f840fff10 msgr2=0x7f7f84108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.828+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 shutdown_connections 2024-04-26T21:09:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:13.829+0000 7f7f83200640 1 -- 172.21.15.27:0/808593847 wait complete. 2024-04-26T21:09:14.599 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:14 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/808593847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:14 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/808593847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:14 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/808593847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:15.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:15 smithi027 ceph-mon[28085]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:15.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:15 smithi204 ceph-mon[32145]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:15.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:15 smithi094 ceph-mon[31273]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:15.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:16.003 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:15 smithi204 podman[62725]: 2024-04-26 21:09:15.766573882 +0000 UTC m=+0.145511230 container create 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T21:09:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 -- 172.21.15.27:0/1444659007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc441042f0 msgr2=0x7fbc4410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 --2- 172.21.15.27:0/1444659007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc441042f0 0x7fbc4410ab80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009ca0 tx=0x7fbc3c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 -- 172.21.15.27:0/1444659007 shutdown_connections 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 --2- 172.21.15.27:0/1444659007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc441042f0 0x7fbc4410ab80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 --2- 172.21.15.27:0/1444659007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc44103930 0x7fbc44103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 --2- 172.21.15.27:0/1444659007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc44102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 -- 172.21.15.27:0/1444659007 >> 172.21.15.27:0/1444659007 conn(0x7fbc440fdcb0 msgr2=0x7fbc441000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 -- 172.21.15.27:0/1444659007 shutdown_connections 2024-04-26T21:09:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.074+0000 7fbc43200640 1 -- 172.21.15.27:0/1444659007 wait complete. 2024-04-26T21:09:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.075+0000 7fbc43200640 1 Processor -- start 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.075+0000 7fbc43200640 1 -- start start 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.075+0000 7fbc43200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc4419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc43200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 0x7fbc4419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc43200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc441042f0 0x7fbc441a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc43200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410e170 con 0x7fbc44103930 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc43200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410dff0 con 0x7fbc441042f0 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc43200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410e2f0 con 0x7fbc44102740 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc41e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc4419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc3be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 0x7fbc4419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc41e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc4419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:59412/0 (socket says 172.21.15.27:59412) 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc41e00640 1 -- 172.21.15.27:0/1210527779 learned_addr learned my addr 172.21.15.27:0/1210527779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.076+0000 7fbc42800640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc441042f0 0x7fbc441a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 -- 172.21.15.27:0/1210527779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 msgr2=0x7fbc4419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc4419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 -- 172.21.15.27:0/1210527779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc441042f0 msgr2=0x7fbc441a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc441042f0 0x7fbc441a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 -- 172.21.15.27:0/1210527779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc441a2660 con 0x7fbc44103930 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3be00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 0x7fbc4419af80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d6d0 tx=0x7fbc2c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c017070 con 0x7fbc44103930 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc2c009820 con 0x7fbc44103930 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c004bc0 con 0x7fbc44103930 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc441a2950 con 0x7fbc44103930 2024-04-26T21:09:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.077+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc441a2ea0 con 0x7fbc44103930 2024-04-26T21:09:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.078+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc441a30d0 con 0x7fbc44103930 2024-04-26T21:09:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.081+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc2c005220 con 0x7fbc44103930 2024-04-26T21:09:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.083+0000 7fbc3aa00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 0x7fbc280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.083+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fbc2c0cfa40 con 0x7fbc44103930 2024-04-26T21:09:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.083+0000 7fbc41e00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 0x7fbc280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.084+0000 7fbc41e00640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 0x7fbc280a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005fd0 tx=0x7fbc3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.088+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbc2c09c1d0 con 0x7fbc44103930 2024-04-26T21:09:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.343+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc441a32a0 con 0x7fbc44103930 2024-04-26T21:09:16.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.344+0000 7fbc3aa00640 1 -- 172.21.15.27:0/1210527779 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fbc2c0a04d0 con 0x7fbc44103930 2024-04-26T21:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:09:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-04-26T21:09:16.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 msgr2=0x7fbc280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 0x7fbc280a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005fd0 tx=0x7fbc3400b410 comp rx=0 tx=0).stop 2024-04-26T21:09:16.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 msgr2=0x7fbc4419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 0x7fbc4419af80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d6d0 tx=0x7fbc2c00db90 comp rx=0 tx=0).stop 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 shutdown_connections 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc280a3bb0 0x7fbc280a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc441042f0 0x7fbc441a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc44103930 0x7fbc4419af80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 --2- 172.21.15.27:0/1210527779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc44102740 0x7fbc4419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.347+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 >> 172.21.15.27:0/1210527779 conn(0x7fbc440fdcb0 msgr2=0x7fbc44074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.348+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 shutdown_connections 2024-04-26T21:09:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:16.348+0000 7fbc43200640 1 -- 172.21.15.27:0/1210527779 wait complete. 2024-04-26T21:09:16.502 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 podman[62725]: 2024-04-26 21:09:16.037244898 +0000 UTC m=+0.416182249 container init 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:09:16.503 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 podman[62725]: 2024-04-26 21:09:16.047915097 +0000 UTC m=+0.426852448 container start 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:09:16.503 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 ceph-mds[62744]: -- 172.21.15.204:0/3637489643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fede8b9a40 con 0x55fede88c800 2024-04-26T21:09:16.503 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: starting mds.c at 2024-04-26T21:09:16.503 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 bash[62725]: 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb 2024-04-26T21:09:16.503 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:09:16 smithi204 systemd[1]: Started Ceph mds.c for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:17.511 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:17.513 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: Deploying daemon mds.h on smithi094 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1210527779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6836/743097112,v1:172.21.15.204:6837/743097112] up:boot 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: fsmap 7 up:standby 2024-04-26T21:09:17.514 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:17 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:09:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: Deploying daemon mds.h on smithi094 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1210527779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6836/743097112,v1:172.21.15.204:6837/743097112] up:boot 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: fsmap 7 up:standby 2024-04-26T21:09:17.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:17 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:09:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: Deploying daemon mds.h on smithi094 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1210527779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6836/743097112,v1:172.21.15.204:6837/743097112] up:boot 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: fsmap 7 up:standby 2024-04-26T21:09:17.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:17 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:09:18.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:18.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- 172.21.15.27:0/175662810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34076970 msgr2=0x7f5c34074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 --2- 172.21.15.27:0/175662810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34076970 0x7f5c34074dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009930 tx=0x7f5c2402a420 comp rx=0 tx=0).stop 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- 172.21.15.27:0/175662810 shutdown_connections 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 --2- 172.21.15.27:0/175662810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34075390 0x7f5c34075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 --2- 172.21.15.27:0/175662810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c3410ed20 0x7f5c34111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 --2- 172.21.15.27:0/175662810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c34076970 0x7f5c34074dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- 172.21.15.27:0/175662810 >> 172.21.15.27:0/175662810 conn(0x7f5c340fdcc0 msgr2=0x7f5c34100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- 172.21.15.27:0/175662810 shutdown_connections 2024-04-26T21:09:18.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- 172.21.15.27:0/175662810 wait complete. 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 Processor -- start 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.598+0000 7f5c39c00640 1 -- start start 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c34075390 0x7f5c3419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34076970 0x7f5c3419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34114770 con 0x7f5c3410ed20 2024-04-26T21:09:18.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341145f0 con 0x7f5c34075390 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341148f0 con 0x7f5c34076970 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41498/0 (socket says 172.21.15.27:41498) 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 -- 172.21.15.27:0/2893276814 learned_addr learned my addr 172.21.15.27:0/2893276814 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c33e00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c34075390 0x7f5c3419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c33400640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34076970 0x7f5c3419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 -- 172.21.15.27:0/2893276814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34076970 msgr2=0x7f5c3419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34076970 0x7f5c3419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 -- 172.21.15.27:0/2893276814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c34075390 msgr2=0x7f5c3419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c34075390 0x7f5c3419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.599+0000 7f5c39200640 1 -- 172.21.15.27:0/2893276814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c341a6af0 con 0x7f5c3410ed20 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c39200640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00eaa0 tx=0x7f5c2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00cdb0 con 0x7f5c3410ed20 2024-04-26T21:09:18.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c2c005020 con 0x7f5c3410ed20 2024-04-26T21:09:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341a6de0 con 0x7f5c3410ed20 2024-04-26T21:09:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0116d0 con 0x7f5c3410ed20 2024-04-26T21:09:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.600+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a72a0 con 0x7f5c3410ed20 2024-04-26T21:09:18.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.601+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c34104ef0 con 0x7f5c3410ed20 2024-04-26T21:09:18.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.604+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5c2c0051d0 con 0x7f5c3410ed20 2024-04-26T21:09:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.605+0000 7f5c30c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 0x7f5c180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.605+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5c2c0d0560 con 0x7f5c3410ed20 2024-04-26T21:09:18.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.605+0000 7f5c33e00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 0x7f5c180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.606+0000 7f5c33e00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 0x7f5c180a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c240057c0 tx=0x7f5c24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.610+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5c2c016030 con 0x7f5c3410ed20 2024-04-26T21:09:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.859+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c3402dad0 con 0x7f5c3410ed20 2024-04-26T21:09:18.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.859+0000 7f5c30c00640 1 -- 172.21.15.27:0/2893276814 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f5c2c09d340 con 0x7f5c3410ed20 2024-04-26T21:09:18.860 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:18.860 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:09:18.860 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.861+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 msgr2=0x7f5c180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.861+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 0x7f5c180a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c240057c0 tx=0x7f5c24000bc0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 msgr2=0x7f5c341a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00eaa0 tx=0x7f5c2c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 shutdown_connections 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5c180a3c30 0x7f5c180a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c34076970 0x7f5c3419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c3410ed20 0x7f5c341a6480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 --2- 172.21.15.27:0/2893276814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c34075390 0x7f5c3419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 >> 172.21.15.27:0/2893276814 conn(0x7f5c340fdcc0 msgr2=0x7f5c341020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 shutdown_connections 2024-04-26T21:09:18.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:18.863+0000 7f5c39c00640 1 -- 172.21.15.27:0/2893276814 wait complete. 2024-04-26T21:09:19.339 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:19 smithi094 ceph-mon[31273]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:19.339 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2893276814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:19.700 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:19 smithi094 systemd[1]: Starting Ceph mds.h for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:09:19.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:19 smithi027 ceph-mon[28085]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2893276814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:19 smithi204 ceph-mon[32145]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2893276814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:19.998 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:19 smithi094 podman[61845]: 2024-04-26 21:09:19.603166051 +0000 UTC m=+0.035437243 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:19.998 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:19 smithi094 podman[61845]: 2024-04-26 21:09:19.716996083 +0000 UTC m=+0.149267307 container create 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:09:19.999 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:19 smithi094 podman[61845]: 2024-04-26 21:09:19.996942097 +0000 UTC m=+0.429213328 container init 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:09:20.276 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:20 smithi094 podman[61845]: 2024-04-26 21:09:20.006603758 +0000 UTC m=+0.438874963 container start 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:09:20.276 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:20 smithi094 ceph-mds[61865]: -- 172.21.15.94:0/1521223273 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d9d431a40 con 0x558d9d405000 2024-04-26T21:09:20.276 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: starting mds.h at 2024-04-26T21:09:20.276 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:20 smithi094 bash[61845]: 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 2024-04-26T21:09:20.276 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:09:20 smithi094 systemd[1]: Started Ceph mds.h for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:20.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.112+0000 7f6375200640 1 -- 172.21.15.27:0/2045481002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63701049b0 msgr2=0x7f6370104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.112+0000 7f6375200640 1 --2- 172.21.15.27:0/2045481002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63701049b0 0x7f6370104dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.112+0000 7f6375200640 1 -- 172.21.15.27:0/2045481002 shutdown_connections 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.112+0000 7f6375200640 1 --2- 172.21.15.27:0/2045481002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6370105ba0 0x7f6370106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.113+0000 7f6375200640 1 --2- 172.21.15.27:0/2045481002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370106560 0x7f637010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.113+0000 7f6375200640 1 --2- 172.21.15.27:0/2045481002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63701049b0 0x7f6370104dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.113+0000 7f6375200640 1 -- 172.21.15.27:0/2045481002 >> 172.21.15.27:0/2045481002 conn(0x7f63700fff40 msgr2=0x7f6370102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.113+0000 7f6375200640 1 -- 172.21.15.27:0/2045481002 shutdown_connections 2024-04-26T21:09:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.113+0000 7f6375200640 1 -- 172.21.15.27:0/2045481002 wait complete. 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 Processor -- start 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 -- start start 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63701049b0 0x7f637019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370105ba0 0x7f637019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63701103b0 con 0x7f6370106560 2024-04-26T21:09:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6370110230 con 0x7f6370105ba0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.114+0000 7f6375200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6370110530 con 0x7f63701049b0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39142/0 (socket says 172.21.15.27:39142) 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 -- 172.21.15.27:0/485223021 learned_addr learned my addr 172.21.15.27:0/485223021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636f400640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63701049b0 0x7f637019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636ea00640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370105ba0 0x7f637019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 -- 172.21.15.27:0/485223021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63701049b0 msgr2=0x7f637019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63701049b0 0x7f637019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 -- 172.21.15.27:0/485223021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370105ba0 msgr2=0x7f637019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370105ba0 0x7f637019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.115+0000 7f636fe00640 1 -- 172.21.15.27:0/485223021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63701a26d0 con 0x7f6370106560 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f636fe00640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f636800eaa0 tx=0x7f636800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800cdb0 con 0x7f6370106560 2024-04-26T21:09:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6368005020 con 0x7f6370106560 2024-04-26T21:09:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63701a29c0 con 0x7f6370106560 2024-04-26T21:09:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680116d0 con 0x7f6370106560 2024-04-26T21:09:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.116+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63701a2f10 con 0x7f6370106560 2024-04-26T21:09:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.117+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637010bd20 con 0x7f6370106560 2024-04-26T21:09:21.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.119+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63680051d0 con 0x7f6370106560 2024-04-26T21:09:21.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.121+0000 7f6367e00640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 0x7f63540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.121+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f63680d0190 con 0x7f6370106560 2024-04-26T21:09:21.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.121+0000 7f636f400640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 0x7f63540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.122+0000 7f636f400640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 0x7f63540a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f63600057c0 tx=0x7f6360005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.125+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6368016030 con 0x7f6370106560 2024-04-26T21:09:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.376+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f637002dad0 con 0x7f6370106560 2024-04-26T21:09:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.378+0000 7f6367e00640 1 -- 172.21.15.27:0/485223021 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f636809cf70 con 0x7f6370106560 2024-04-26T21:09:21.379 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:21.379 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2024-04-26T21:09:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 msgr2=0x7f63540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 0x7f63540a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f63600057c0 tx=0x7f6360005a20 comp rx=0 tx=0).stop 2024-04-26T21:09:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 msgr2=0x7f63701a2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f636800eaa0 tx=0x7f636800ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 shutdown_connections 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63540a3c30 0x7f63540a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6370106560 0x7f63701a2060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6370105ba0 0x7f637019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 --2- 172.21.15.27:0/485223021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63701049b0 0x7f637019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 >> 172.21.15.27:0/485223021 conn(0x7f63700fff40 msgr2=0x7f6370108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 shutdown_connections 2024-04-26T21:09:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:21.381+0000 7f6375200640 1 -- 172.21.15.27:0/485223021 wait complete. 2024-04-26T21:09:21.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:boot 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: fsmap 8 up:standby 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:21.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:21 smithi204 ceph-mon[32145]: Deploying daemon mds.i on smithi204 2024-04-26T21:09:21.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:21.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:boot 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: fsmap 8 up:standby 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:21.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:21 smithi094 ceph-mon[31273]: Deploying daemon mds.i on smithi204 2024-04-26T21:09:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:boot 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: fsmap 8 up:standby 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:09:21.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:21.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:21.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:21.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:21.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:21 smithi027 ceph-mon[28085]: Deploying daemon mds.i on smithi204 2024-04-26T21:09:22.427 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:22 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/485223021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:22.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:22 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/485223021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:22 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/485223021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:23.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:23.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:23 smithi204 ceph-mon[32145]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:23.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:23 smithi094 ceph-mon[31273]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:23.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 -- 172.21.15.27:0/3760062121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224108450 msgr2=0x7f422410a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 --2- 172.21.15.27:0/3760062121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224108450 0x7f422410a8a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 -- 172.21.15.27:0/3760062121 shutdown_connections 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 --2- 172.21.15.27:0/3760062121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224105b20 0x7f4224107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 --2- 172.21.15.27:0/3760062121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224108450 0x7f422410a8a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 --2- 172.21.15.27:0/3760062121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42241031f0 0x7f42241055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 -- 172.21.15.27:0/3760062121 >> 172.21.15.27:0/3760062121 conn(0x7f42240fcbc0 msgr2=0x7f42240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 -- 172.21.15.27:0/3760062121 shutdown_connections 2024-04-26T21:09:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.631+0000 7f4223e00640 1 -- 172.21.15.27:0/3760062121 wait complete. 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.632+0000 7f4223e00640 1 Processor -- start 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.632+0000 7f4223e00640 1 -- start start 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.632+0000 7f4223e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42241031f0 0x7f4224102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.632+0000 7f4223e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 0x7f42241016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f422410e1a0 con 0x7f4224105b20 2024-04-26T21:09:23.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f422410e020 con 0x7f4224108450 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f422410e320 con 0x7f42241031f0 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39168/0 (socket says 172.21.15.27:39168) 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 -- 172.21.15.27:0/1933453337 learned_addr learned my addr 172.21.15.27:0/1933453337 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4222a00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42241031f0 0x7f4224102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 -- 172.21.15.27:0/1933453337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42241031f0 msgr2=0x7f4224102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42241031f0 0x7f4224102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223400640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 0x7f42241016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 -- 172.21.15.27:0/1933453337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 msgr2=0x7f42241016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 0x7f42241016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f421a000640 1 -- 172.21.15.27:0/1933453337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4224101f70 con 0x7f4224105b20 2024-04-26T21:09:23.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.633+0000 7f4223400640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 0x7f42241016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f421a000640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c970 tx=0x7f420c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c014070 con 0x7f4224105b20 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420c004440 con 0x7f4224105b20 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c003d40 con 0x7f4224105b20 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42241a28d0 con 0x7f4224105b20 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.634+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42241a2e70 con 0x7f4224105b20 2024-04-26T21:09:23.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.635+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42240fe6b0 con 0x7f4224105b20 2024-04-26T21:09:23.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.638+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f420c017070 con 0x7f4224105b20 2024-04-26T21:09:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.639+0000 7f421be00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 0x7f42080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.639+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f420c0cff30 con 0x7f4224105b20 2024-04-26T21:09:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.639+0000 7f4222a00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 0x7f42080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.640+0000 7f4222a00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 0x7f42080a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4214005fd0 tx=0x7f4214005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.644+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f420c01e020 con 0x7f4224105b20 2024-04-26T21:09:23.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:23 smithi027 ceph-mon[28085]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:23.838 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 podman[63028]: 2024-04-26 21:09:23.83168067 +0000 UTC m=+0.374781793 container init b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T21:09:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.894+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f422402dad0 con 0x7f4224105b20 2024-04-26T21:09:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.896+0000 7f421be00640 1 -- 172.21.15.27:0/1933453337 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f420c09cd10 con 0x7f4224105b20 2024-04-26T21:09:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:23.897 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.898+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 msgr2=0x7f42080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.898+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 0x7f42080a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4214005fd0 tx=0x7f4214005850 comp rx=0 tx=0).stop 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.898+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 msgr2=0x7f42241011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.898+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c970 tx=0x7f420c00ce30 comp rx=0 tx=0).stop 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 shutdown_connections 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f42080a3c30 0x7f42080a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4224108450 0x7f42241016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4224105b20 0x7f42241011a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 --2- 172.21.15.27:0/1933453337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42241031f0 0x7f4224102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 >> 172.21.15.27:0/1933453337 conn(0x7f42240fcbc0 msgr2=0x7f4224103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 shutdown_connections 2024-04-26T21:09:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:23.899+0000 7f4223e00640 1 -- 172.21.15.27:0/1933453337 wait complete. 2024-04-26T21:09:24.252 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 podman[63028]: 2024-04-26 21:09:23.84196868 +0000 UTC m=+0.385069785 container start b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:09:24.252 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 ceph-mds[63047]: -- 172.21.15.204:0/2914789796 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de96429a40 con 0x55de963fc800 2024-04-26T21:09:24.252 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: starting mds.i at 2024-04-26T21:09:24.253 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 bash[63028]: b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 2024-04-26T21:09:24.253 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:09:23 smithi204 systemd[1]: Started Ceph mds.i for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1933453337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:24 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1933453337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:24 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1933453337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:24 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:25 smithi027 ceph-mon[28085]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:25 smithi027 ceph-mon[28085]: Deploying daemon mds.g on smithi027 2024-04-26T21:09:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:25 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6838/1122553339,v1:172.21.15.204:6839/1122553339] up:boot 2024-04-26T21:09:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:25 smithi027 ceph-mon[28085]: fsmap 9 up:standby 2024-04-26T21:09:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:25 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:09:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:25 smithi094 ceph-mon[31273]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:25 smithi094 ceph-mon[31273]: Deploying daemon mds.g on smithi027 2024-04-26T21:09:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:25 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6838/1122553339,v1:172.21.15.204:6839/1122553339] up:boot 2024-04-26T21:09:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:25 smithi094 ceph-mon[31273]: fsmap 9 up:standby 2024-04-26T21:09:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:25 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:09:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:25 smithi204 ceph-mon[32145]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:25 smithi204 ceph-mon[32145]: Deploying daemon mds.g on smithi027 2024-04-26T21:09:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:25 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6838/1122553339,v1:172.21.15.204:6839/1122553339] up:boot 2024-04-26T21:09:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:25 smithi204 ceph-mon[32145]: fsmap 9 up:standby 2024-04-26T21:09:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:25 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:09:25.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.145+0000 7f6ef7200640 1 -- 172.21.15.27:0/316683518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef81039a0 msgr2=0x7f6ef8103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 --2- 172.21.15.27:0/316683518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef81039a0 0x7f6ef8103e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 -- 172.21.15.27:0/316683518 shutdown_connections 2024-04-26T21:09:26.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 --2- 172.21.15.27:0/316683518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8104360 0x7f6ef810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 --2- 172.21.15.27:0/316683518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef81039a0 0x7f6ef8103e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 --2- 172.21.15.27:0/316683518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81027b0 0x7f6ef8102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 -- 172.21.15.27:0/316683518 >> 172.21.15.27:0/316683518 conn(0x7f6ef80fdd40 msgr2=0x7f6ef8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 -- 172.21.15.27:0/316683518 shutdown_connections 2024-04-26T21:09:26.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.146+0000 7f6ef7200640 1 -- 172.21.15.27:0/316683518 wait complete. 2024-04-26T21:09:26.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 Processor -- start 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 -- start start 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef81027b0 0x7f6ef80766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81039a0 0x7f6ef8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810e1e0 con 0x7f6ef8104360 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810e060 con 0x7f6ef81039a0 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.147+0000 7f6ef7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810e360 con 0x7f6ef81027b0 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39182/0 (socket says 172.21.15.27:39182) 2024-04-26T21:09:26.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 -- 172.21.15.27:0/1202774769 learned_addr learned my addr 172.21.15.27:0/1202774769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef5400640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81039a0 0x7f6ef8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef5e00640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef81027b0 0x7f6ef80766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 -- 172.21.15.27:0/1202774769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef81027b0 msgr2=0x7f6ef80766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef81027b0 0x7f6ef80766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 -- 172.21.15.27:0/1202774769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81039a0 msgr2=0x7f6ef8074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81039a0 0x7f6ef8074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.148+0000 7f6ef6800640 1 -- 172.21.15.27:0/1202774769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8075af0 con 0x7f6ef8104360 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6ef6800640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000eaa0 tx=0x7f6ef000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000cdb0 con 0x7f6ef8104360 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ef0005020 con 0x7f6ef8104360 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef81a28f0 con 0x7f6ef8104360 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00116d0 con 0x7f6ef8104360 2024-04-26T21:09:26.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.149+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a2e10 con 0x7f6ef8104360 2024-04-26T21:09:26.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.150+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8069670 con 0x7f6ef8104360 2024-04-26T21:09:26.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.152+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ef00051d0 con 0x7f6ef8104360 2024-04-26T21:09:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.154+0000 7f6eeea00640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 0x7f6edc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.154+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6ef00d02d0 con 0x7f6ef8104360 2024-04-26T21:09:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.154+0000 7f6ef5e00640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 0x7f6edc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.155+0000 7f6ef5e00640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 0x7f6edc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80058c0 tx=0x7f6ee8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.159+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ef0016030 con 0x7f6ef8104360 2024-04-26T21:09:26.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.427+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef8063530 con 0x7f6ef8104360 2024-04-26T21:09:26.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.428+0000 7f6eeea00640 1 -- 172.21.15.27:0/1202774769 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7f6ef009d0b0 con 0x7f6ef8104360 2024-04-26T21:09:26.429 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:26.429 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:26.430 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-04-26T21:09:26.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 msgr2=0x7f6edc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 0x7f6edc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80058c0 tx=0x7f6ee8005850 comp rx=0 tx=0).stop 2024-04-26T21:09:26.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 msgr2=0x7f6ef8075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000eaa0 tx=0x7f6ef000ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 shutdown_connections 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6edc0a3bb0 0x7f6edc0a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ef8104360 0x7f6ef8075260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef81039a0 0x7f6ef8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 --2- 172.21.15.27:0/1202774769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef81027b0 0x7f6ef80766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 >> 172.21.15.27:0/1202774769 conn(0x7f6ef80fdd40 msgr2=0x7f6ef8106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.431+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 shutdown_connections 2024-04-26T21:09:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:26.432+0000 7f6ef7200640 1 -- 172.21.15.27:0/1202774769 wait complete. 2024-04-26T21:09:27.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:27 smithi027 ceph-mon[28085]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:27.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:27 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1202774769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:27 smithi204 ceph-mon[32145]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:27 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1202774769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:27 smithi094 ceph-mon[31273]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:27 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1202774769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:27.849 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 podman[69622]: 2024-04-26 21:09:27.56085614 +0000 UTC m=+0.146284844 container create 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:09:27.849 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 podman[69622]: 2024-04-26 21:09:27.847641863 +0000 UTC m=+0.433070522 container init 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T21:09:28.234 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 podman[69622]: 2024-04-26 21:09:27.856606363 +0000 UTC m=+0.442035030 container start 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:09:28.234 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 ceph-mds[69648]: -- 172.21.15.27:0/2816458930 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdeb76da40 con 0x55cdeb741000 2024-04-26T21:09:28.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: starting mds.g at 2024-04-26T21:09:28.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 bash[69622]: 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 2024-04-26T21:09:28.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:09:27 smithi027 systemd[1]: Started Ceph mds.g for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:28.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.662+0000 7f0acbe00640 1 -- 172.21.15.27:0/2436965999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc103950 msgr2=0x7f0acc105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.662+0000 7f0acbe00640 1 --2- 172.21.15.27:0/2436965999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc103950 0x7f0acc105dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009900 tx=0x7f0ab802a410 comp rx=0 tx=0).stop 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 -- 172.21.15.27:0/2436965999 shutdown_connections 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 --2- 172.21.15.27:0/2436965999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc106300 0x7f0acc106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 --2- 172.21.15.27:0/2436965999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc103950 0x7f0acc105dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 --2- 172.21.15.27:0/2436965999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc100fe0 0x7f0acc103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 -- 172.21.15.27:0/2436965999 >> 172.21.15.27:0/2436965999 conn(0x7f0acc0fcbc0 msgr2=0x7f0acc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 -- 172.21.15.27:0/2436965999 shutdown_connections 2024-04-26T21:09:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.663+0000 7f0acbe00640 1 -- 172.21.15.27:0/2436965999 wait complete. 2024-04-26T21:09:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.664+0000 7f0acbe00640 1 Processor -- start 2024-04-26T21:09:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.664+0000 7f0acbe00640 1 -- start start 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc103950 0x7f0acc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc106300 0x7f0acc1a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc114770 con 0x7f0acc100fe0 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1145f0 con 0x7f0acc103950 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1148f0 con 0x7f0acc106300 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39210/0 (socket says 172.21.15.27:39210) 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acaa00640 1 -- 172.21.15.27:0/637121471 learned_addr learned my addr 172.21.15.27:0/637121471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0aca000640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc103950 0x7f0acc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.665+0000 7f0acb400640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc106300 0x7f0acc1a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 -- 172.21.15.27:0/637121471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc106300 msgr2=0x7f0acc1a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc106300 0x7f0acc1a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 -- 172.21.15.27:0/637121471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc103950 msgr2=0x7f0acc19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc103950 0x7f0acc19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 -- 172.21.15.27:0/637121471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc1a6ac0 con 0x7f0acc100fe0 2024-04-26T21:09:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0acaa00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00d950 tx=0x7f0abc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.666+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc017070 con 0x7f0acc100fe0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.667+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0abc009c30 con 0x7f0acc100fe0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.667+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1a6db0 con 0x7f0acc100fe0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.667+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00f620 con 0x7f0acc100fe0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.667+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1a7210 con 0x7f0acc100fe0 2024-04-26T21:09:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.668+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc0fe6b0 con 0x7f0acc100fe0 2024-04-26T21:09:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.670+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0abc010460 con 0x7f0acc100fe0 2024-04-26T21:09:28.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.672+0000 7f0ac3400640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 0x7f0ab00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.673+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0abc0cfcc0 con 0x7f0acc100fe0 2024-04-26T21:09:28.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.673+0000 7f0aca000640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 0x7f0ab00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.674+0000 7f0aca000640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 0x7f0ab00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8006010 tx=0x7f0ab8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.678+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0abc0d6080 con 0x7f0acc100fe0 2024-04-26T21:09:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.928+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0acc02dad0 con 0x7f0acc100fe0 2024-04-26T21:09:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.929+0000 7f0ac3400640 1 -- 172.21.15.27:0/637121471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f0abc09caa0 con 0x7f0acc100fe0 2024-04-26T21:09:28.930 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-04-26T21:09:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 msgr2=0x7f0ab00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 0x7f0ab00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8006010 tx=0x7f0ab8031400 comp rx=0 tx=0).stop 2024-04-26T21:09:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 msgr2=0x7f0acc19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00d950 tx=0x7f0abc00de10 comp rx=0 tx=0).stop 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 shutdown_connections 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0ab00a3bb0 0x7f0ab00a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0acc106300 0x7f0acc1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0acc103950 0x7f0acc19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 --2- 172.21.15.27:0/637121471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0acc100fe0 0x7f0acc19eea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 >> 172.21.15.27:0/637121471 conn(0x7f0acc0fcbc0 msgr2=0x7f0acc101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 shutdown_connections 2024-04-26T21:09:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:28.931+0000 7f0acbe00640 1 -- 172.21.15.27:0/637121471 wait complete. 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: Deploying daemon mds.a on smithi027 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6838/4118119230,v1:172.21.15.27:6839/4118119230] up:boot 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: fsmap 10 up:standby 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:09:29.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:29 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/637121471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: Deploying daemon mds.a on smithi027 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6838/4118119230,v1:172.21.15.27:6839/4118119230] up:boot 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: fsmap 10 up:standby 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:09:29.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:29 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/637121471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: Deploying daemon mds.a on smithi027 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6838/4118119230,v1:172.21.15.27:6839/4118119230] up:boot 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: fsmap 10 up:standby 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:09:29.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:29 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/637121471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:30.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:31.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:31 smithi027 ceph-mon[28085]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:31.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 -- 172.21.15.27:0/2257361543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 msgr2=0x7fc4881093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 --2- 172.21.15.27:0/2257361543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc4881093b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc480009ca0 tx=0x7fc48002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 -- 172.21.15.27:0/2257361543 shutdown_connections 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 --2- 172.21.15.27:0/2257361543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810a180 0x7fc48810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 --2- 172.21.15.27:0/2257361543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810ab40 0x7fc4881113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 --2- 172.21.15.27:0/2257361543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc4881093b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 -- 172.21.15.27:0/2257361543 >> 172.21.15.27:0/2257361543 conn(0x7fc488077f40 msgr2=0x7fc48807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 -- 172.21.15.27:0/2257361543 shutdown_connections 2024-04-26T21:09:31.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.294+0000 7fc48ca00640 1 -- 172.21.15.27:0/2257361543 wait complete. 2024-04-26T21:09:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.295+0000 7fc48ca00640 1 Processor -- start 2024-04-26T21:09:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.295+0000 7fc48ca00640 1 -- start start 2024-04-26T21:09:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 0x7fc48819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810ab40 0x7fc4881a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881149d0 con 0x7fc488108f90 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc488114850 con 0x7fc48810a180 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc48ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc488114b50 con 0x7fc48810ab40 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc485e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc485e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46490/0 (socket says 172.21.15.27:46490) 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc485e00640 1 -- 172.21.15.27:0/1449793098 learned_addr learned my addr 172.21.15.27:0/1449793098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc486800640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810ab40 0x7fc4881a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.296+0000 7fc485400640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 0x7fc48819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 -- 172.21.15.27:0/1449793098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810ab40 msgr2=0x7fc4881a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810ab40 0x7fc4881a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 -- 172.21.15.27:0/1449793098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 msgr2=0x7fc48819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 0x7fc48819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 -- 172.21.15.27:0/1449793098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4881a6c60 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485400640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 0x7fc48819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc485e00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc48002aeb0 tx=0x7fc480003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480004240 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc480003e40 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4881a6ef0 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48003b410 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.297+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4881a7350 con 0x7fc488108f90 2024-04-26T21:09:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.299+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4881102d0 con 0x7fc488108f90 2024-04-26T21:09:31.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.301+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4800305c0 con 0x7fc488108f90 2024-04-26T21:09:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.303+0000 7fc47ea00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 0x7fc46c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.303+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc4800f18c0 con 0x7fc488108f90 2024-04-26T21:09:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.304+0000 7fc485400640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 0x7fc46c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.304+0000 7fc485400640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 0x7fc46c0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4881a0550 tx=0x7fc470005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:31.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.307+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc4800be6a0 con 0x7fc488108f90 2024-04-26T21:09:31.484 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:31 smithi027 systemd[1]: Starting Ceph mds.a for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:09:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:31 smithi204 ceph-mon[32145]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:31.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:31 smithi094 ceph-mon[31273]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.589+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc48802dad0 con 0x7fc488108f90 2024-04-26T21:09:31.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.591+0000 7fc47ea00640 1 -- 172.21.15.27:0/1449793098 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fc4800bdfa0 con 0x7fc488108f90 2024-04-26T21:09:31.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:31.593 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:31.593 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.594+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 msgr2=0x7fc46c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.594+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 0x7fc46c0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4881a0550 tx=0x7fc470005d90 comp rx=0 tx=0).stop 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.594+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 msgr2=0x7fc48819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.594+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc48002aeb0 tx=0x7fc480003c20 comp rx=0 tx=0).stop 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 shutdown_connections 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc46c0a3bb0 0x7fc46c0a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc48810ab40 0x7fc4881a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc48810a180 0x7fc48819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 --2- 172.21.15.27:0/1449793098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488108f90 0x7fc48819f040 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 >> 172.21.15.27:0/1449793098 conn(0x7fc488077f40 msgr2=0x7fc48810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:31.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 shutdown_connections 2024-04-26T21:09:31.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:31.595+0000 7fc48ca00640 1 -- 172.21.15.27:0/1449793098 wait complete. 2024-04-26T21:09:31.986 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:31 smithi027 podman[70011]: 2024-04-26 21:09:31.486545279 +0000 UTC m=+0.053456740 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:09:31.986 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:31 smithi027 podman[70011]: 2024-04-26 21:09:31.598072887 +0000 UTC m=+0.164984351 container create 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:32 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1449793098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 podman[70011]: 2024-04-26 21:09:32.085918377 +0000 UTC m=+0.652829892 container init 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 podman[70011]: 2024-04-26 21:09:32.100135191 +0000 UTC m=+0.667046653 container start 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 ceph-mds[70032]: -- 172.21.15.27:0/4037224620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b80f4b9a40 con 0x55b80f48d000 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: starting mds.a at 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 bash[70011]: 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 2024-04-26T21:09:32.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:09:32 smithi027 systemd[1]: Started Ceph mds.a for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:32 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1449793098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:32.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:32 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1449793098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: Deploying daemon mds.b on smithi094 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:boot 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: fsmap 11 up:standby 2024-04-26T21:09:33.569 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:33 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:09:33.619 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: Deploying daemon mds.b on smithi094 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:boot 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: fsmap 11 up:standby 2024-04-26T21:09:33.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:33 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:09:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: Deploying daemon mds.b on smithi094 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:boot 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: fsmap 11 up:standby 2024-04-26T21:09:33.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:33 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:09:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 -- 172.21.15.27:0/1945062821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc105d50 msgr2=0x7f60bc108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1945062821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc105d50 0x7f60bc108140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a410 comp rx=0 tx=0).stop 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 -- 172.21.15.27:0/1945062821 shutdown_connections 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1945062821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 0x7f60bc10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1945062821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc105d50 0x7f60bc108140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1945062821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc103420 0x7f60bc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 -- 172.21.15.27:0/1945062821 >> 172.21.15.27:0/1945062821 conn(0x7f60bc0fce50 msgr2=0x7f60bc0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 -- 172.21.15.27:0/1945062821 shutdown_connections 2024-04-26T21:09:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.847+0000 7f60c2c00640 1 -- 172.21.15.27:0/1945062821 wait complete. 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.848+0000 7f60c2c00640 1 Processor -- start 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 -- start start 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 0x7f60bc19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 0x7f60bc1a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10e3d0 con 0x7f60bc103420 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10e250 con 0x7f60bc105d50 2024-04-26T21:09:33.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60c2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10e550 con 0x7f60bc108680 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46524/0 (socket says 172.21.15.27:46524) 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60bbe00640 1 -- 172.21.15.27:0/1883023182 learned_addr learned my addr 172.21.15.27:0/1883023182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.849+0000 7f60bb400640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 0x7f60bc19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60c0c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 0x7f60bc1a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 -- 172.21.15.27:0/1883023182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 msgr2=0x7f60bc1a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 0x7f60bc1a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 -- 172.21.15.27:0/1883023182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 msgr2=0x7f60bc19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 0x7f60bc19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 -- 172.21.15.27:0/1883023182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc1a2920 con 0x7f60bc103420 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bb400640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 0x7f60bc19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60bbe00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00d950 tx=0x7f60ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac017070 con 0x7f60bc103420 2024-04-26T21:09:33.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60ac009c30 con 0x7f60bc103420 2024-04-26T21:09:33.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc1a2c10 con 0x7f60bc103420 2024-04-26T21:09:33.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.850+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00f620 con 0x7f60bc103420 2024-04-26T21:09:33.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.851+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc1a31b0 con 0x7f60bc103420 2024-04-26T21:09:33.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.852+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc10a1d0 con 0x7f60bc103420 2024-04-26T21:09:33.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.854+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60ac005030 con 0x7f60bc103420 2024-04-26T21:09:33.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.855+0000 7f60b8c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.855+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f60ac0cfcc0 con 0x7f60bc103420 2024-04-26T21:09:33.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.856+0000 7f60bb400640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.856+0000 7f60bb400640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 0x7f60a00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60a80057c0 tx=0x7f60a8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:33.858+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f60ac09cb20 con 0x7f60bc103420 2024-04-26T21:09:34.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.105+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60bc02dd60 con 0x7f60bc103420 2024-04-26T21:09:34.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.106+0000 7f60b8c00640 1 -- 172.21.15.27:0/1883023182 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f60ac009de0 con 0x7f60bc103420 2024-04-26T21:09:34.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:34.108 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:09:34.108 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-04-26T21:09:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 msgr2=0x7f60a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 0x7f60a00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60a80057c0 tx=0x7f60a8031400 comp rx=0 tx=0).stop 2024-04-26T21:09:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 msgr2=0x7f60bc19ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00d950 tx=0x7f60ac00de10 comp rx=0 tx=0).stop 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 shutdown_connections 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60bc108680 0x7f60bc1a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60bc105d50 0x7f60bc19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 --2- 172.21.15.27:0/1883023182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60bc103420 0x7f60bc19ad10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.109+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 >> 172.21.15.27:0/1883023182 conn(0x7f60bc0fce50 msgr2=0x7f60bc102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.110+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 shutdown_connections 2024-04-26T21:09:34.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:34.110+0000 7f60c2c00640 1 -- 172.21.15.27:0/1883023182 wait complete. 2024-04-26T21:09:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:34 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1883023182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:34 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1883023182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:34 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1883023182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:35.405 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:35 smithi094 ceph-mon[31273]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:35 smithi027 ceph-mon[28085]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:35.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:35 smithi204 ceph-mon[32145]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:36.027 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:35 smithi094 podman[62158]: 2024-04-26 21:09:35.790325304 +0000 UTC m=+0.148577477 container create c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T21:09:36.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:36.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 -- 172.21.15.27:0/2018268580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed81033f0 msgr2=0x7faed81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 --2- 172.21.15.27:0/2018268580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed81033f0 0x7faed81057e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faec8009ca0 tx=0x7faec802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 -- 172.21.15.27:0/2018268580 shutdown_connections 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 --2- 172.21.15.27:0/2018268580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed8108650 0x7faed810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 --2- 172.21.15.27:0/2018268580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 0x7faed8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 --2- 172.21.15.27:0/2018268580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed81033f0 0x7faed81057e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 -- 172.21.15.27:0/2018268580 >> 172.21.15.27:0/2018268580 conn(0x7faed80fcde0 msgr2=0x7faed80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 -- 172.21.15.27:0/2018268580 shutdown_connections 2024-04-26T21:09:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.363+0000 7faedda00640 1 -- 172.21.15.27:0/2018268580 wait complete. 2024-04-26T21:09:36.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.364+0000 7faedda00640 1 Processor -- start 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.364+0000 7faedda00640 1 -- start start 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed81033f0 0x7faed819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 0x7faed819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faed810e3c0 con 0x7faed8108650 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faed810e240 con 0x7faed8105d20 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faedda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faed810e540 con 0x7faed81033f0 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46550/0 (socket says 172.21.15.27:46550) 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed7400640 1 -- 172.21.15.27:0/334581286 learned_addr learned my addr 172.21.15.27:0/334581286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed6a00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed81033f0 0x7faed819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed6000640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 0x7faed819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.365+0000 7faed7400640 1 -- 172.21.15.27:0/334581286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed81033f0 msgr2=0x7faed819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faed7400640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed81033f0 0x7faed819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faed7400640 1 -- 172.21.15.27:0/334581286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 msgr2=0x7faed819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faed7400640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 0x7faed819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faed7400640 1 -- 172.21.15.27:0/334581286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed81a2960 con 0x7faed8108650 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faed7400640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faed000d750 tx=0x7faed000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017070 con 0x7faed8108650 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faed00054b0 con 0x7faed8108650 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed81a2c50 con 0x7faed8108650 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0004b90 con 0x7faed8108650 2024-04-26T21:09:36.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.366+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a31a0 con 0x7faed8108650 2024-04-26T21:09:36.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.368+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed810a1a0 con 0x7faed8108650 2024-04-26T21:09:36.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.369+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faed0004d30 con 0x7faed8108650 2024-04-26T21:09:36.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.371+0000 7faecf400640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 0x7faebc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.371+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7faed00cfad0 con 0x7faed8108650 2024-04-26T21:09:36.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.371+0000 7faed6a00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 0x7faebc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.372+0000 7faed6a00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 0x7faebc0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faec8000f80 tx=0x7faec8031450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.376+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faed009c8b0 con 0x7faed8108650 2024-04-26T21:09:36.526 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 podman[62158]: 2024-04-26 21:09:36.078079294 +0000 UTC m=+0.436331510 container init c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T21:09:36.527 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 podman[62158]: 2024-04-26 21:09:36.089445691 +0000 UTC m=+0.447697888 container start c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T21:09:36.527 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 ceph-mds[62177]: -- 172.21.15.94:0/4001345916 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e263e6ba40 con 0x55e263e3e800 2024-04-26T21:09:36.527 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: starting mds.b at 2024-04-26T21:09:36.527 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 bash[62158]: c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a 2024-04-26T21:09:36.527 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:36 smithi094 systemd[1]: Started Ceph mds.b for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:09:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.629+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faed80635d0 con 0x7faed8108650 2024-04-26T21:09:36.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.631+0000 7faecf400640 1 -- 172.21.15.27:0/334581286 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7faed009c1b0 con 0x7faed8108650 2024-04-26T21:09:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:36.632 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:09:36.632 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.633+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 msgr2=0x7faebc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.633+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 0x7faebc0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faec8000f80 tx=0x7faec8031450 comp rx=0 tx=0).stop 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.633+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 msgr2=0x7faed81a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.633+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faed000d750 tx=0x7faed000dc10 comp rx=0 tx=0).stop 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 shutdown_connections 2024-04-26T21:09:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faebc0a3c30 0x7faebc0a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed8108650 0x7faed81a22c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed8105d20 0x7faed819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 --2- 172.21.15.27:0/334581286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed81033f0 0x7faed819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 >> 172.21.15.27:0/334581286 conn(0x7faed80fcde0 msgr2=0x7faed8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 shutdown_connections 2024-04-26T21:09:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.634+0000 7faedda00640 1 -- 172.21.15.27:0/334581286 wait complete. 2024-04-26T21:09:36.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 -- 172.21.15.27:0/607407586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c103940 msgr2=0x7f786c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 --2- 172.21.15.27:0/607407586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c103940 0x7f786c103dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7858009900 tx=0x7f785802a410 comp rx=0 tx=0).stop 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 -- 172.21.15.27:0/607407586 shutdown_connections 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 --2- 172.21.15.27:0/607407586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c104300 0x7f786c10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 --2- 172.21.15.27:0/607407586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c103940 0x7f786c103dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 --2- 172.21.15.27:0/607407586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 0x7f786c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 -- 172.21.15.27:0/607407586 >> 172.21.15.27:0/607407586 conn(0x7f786c0fdcc0 msgr2=0x7f786c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 -- 172.21.15.27:0/607407586 shutdown_connections 2024-04-26T21:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.876+0000 7f786be00640 1 -- 172.21.15.27:0/607407586 wait complete. 2024-04-26T21:09:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 Processor -- start 2024-04-26T21:09:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 -- start start 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 0x7f786c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c103940 0x7f786c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f786c10e160 con 0x7f786c104300 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f786c10dfe0 con 0x7f786c102750 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f786c10e2e0 con 0x7f786c103940 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.878+0000 7f786b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46570/0 (socket says 172.21.15.27:46570) 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 -- 172.21.15.27:0/310875843 learned_addr learned my addr 172.21.15.27:0/310875843 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786aa00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 0x7f786c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786a000640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c103940 0x7f786c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 -- 172.21.15.27:0/310875843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c103940 msgr2=0x7f786c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c103940 0x7f786c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 -- 172.21.15.27:0/310875843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 msgr2=0x7f786c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 0x7f786c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 -- 172.21.15.27:0/310875843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c1a2690 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.879+0000 7f786b400640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f786400cce0 tx=0x7f786400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.880+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786400f450 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.880+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f786400fa80 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.880+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786c1a2980 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.880+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864018ad0 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.880+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786c1a2e50 con 0x7f786c104300 2024-04-26T21:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.881+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786c1a3080 con 0x7f786c104300 2024-04-26T21:09:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.884+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f786400f5f0 con 0x7f786c104300 2024-04-26T21:09:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.886+0000 7f7863400640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 0x7f78500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.886+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f78640d07d0 con 0x7f786c104300 2024-04-26T21:09:36.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.886+0000 7f786aa00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 0x7f78500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.887+0000 7f786aa00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 0x7f78500a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f785c005fd0 tx=0x7f785c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:36.891+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7864014030 con 0x7f786c104300 2024-04-26T21:09:37.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.108+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f786c1a32f0 con 0x7f786c104300 2024-04-26T21:09:37.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.111+0000 7f7863400640 1 -- 172.21.15.27:0/310875843 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7f786409d650 con 0x7f786c104300 2024-04-26T21:09:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:37.113 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:08:37.469170+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.114+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 msgr2=0x7f78500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.114+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 0x7f78500a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f785c005fd0 tx=0x7f785c005850 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.114+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 msgr2=0x7f786c1a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.114+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f786400cce0 tx=0x7f786400a620 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 shutdown_connections 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f78500a3bb0 0x7f78500a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c103940 0x7f786c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f786c104300 0x7f786c1a2020 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 --2- 172.21.15.27:0/310875843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f786c102750 0x7f786c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 >> 172.21.15.27:0/310875843 conn(0x7f786c0fdcc0 msgr2=0x7f786c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 shutdown_connections 2024-04-26T21:09:37.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.115+0000 7f786be00640 1 -- 172.21.15.27:0/310875843 wait complete. 2024-04-26T21:09:37.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:37.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 -- 172.21.15.27:0/726965789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd24102930 msgr2=0x7fcd24102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 --2- 172.21.15.27:0/726965789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd24102930 0x7fcd24102db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009900 tx=0x7fcd1002a410 comp rx=0 tx=0).stop 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 -- 172.21.15.27:0/726965789 shutdown_connections 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 --2- 172.21.15.27:0/726965789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd241032f0 0x7fcd24109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 --2- 172.21.15.27:0/726965789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd24102930 0x7fcd24102db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 --2- 172.21.15.27:0/726965789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24101740 0x7fcd24101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 -- 172.21.15.27:0/726965789 >> 172.21.15.27:0/726965789 conn(0x7fcd240fccd0 msgr2=0x7fcd240ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 -- 172.21.15.27:0/726965789 shutdown_connections 2024-04-26T21:09:37.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.358+0000 7fcd29800640 1 -- 172.21.15.27:0/726965789 wait complete. 2024-04-26T21:09:37.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.360+0000 7fcd29800640 1 Processor -- start 2024-04-26T21:09:37.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.360+0000 7fcd29800640 1 -- start start 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd24101740 0x7fcd2419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24102930 0x7fcd2419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410e240 con 0x7fcd241032f0 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410e0c0 con 0x7fcd24102930 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd29800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410e3c0 con 0x7fcd24101740 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46592/0 (socket says 172.21.15.27:46592) 2024-04-26T21:09:37.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 -- 172.21.15.27:0/1430349363 learned_addr learned my addr 172.21.15.27:0/1430349363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:37.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd23e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd24101740 0x7fcd2419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd23400640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24102930 0x7fcd2419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 -- 172.21.15.27:0/1430349363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd24101740 msgr2=0x7fcd2419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd24101740 0x7fcd2419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 -- 172.21.15.27:0/1430349363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24102930 msgr2=0x7fcd2419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24102930 0x7fcd2419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.361+0000 7fcd28e00640 1 -- 172.21.15.27:0/1430349363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a27e0 con 0x7fcd241032f0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd28e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00eaa0 tx=0x7fcd1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00cdb0 con 0x7fcd241032f0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c005020 con 0x7fcd241032f0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0116d0 con 0x7fcd241032f0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a2ad0 con 0x7fcd241032f0 2024-04-26T21:09:37.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.362+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a2fa0 con 0x7fcd241032f0 2024-04-26T21:09:37.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.364+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd24069670 con 0x7fcd241032f0 2024-04-26T21:09:37.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.365+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd1c0051d0 con 0x7fcd241032f0 2024-04-26T21:09:37.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.367+0000 7fcd20c00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 0x7fcd080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.367+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcd1c0d01c0 con 0x7fcd241032f0 2024-04-26T21:09:37.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.367+0000 7fcd23e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 0x7fcd080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.368+0000 7fcd23e00640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 0x7fcd080a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd24102790 tx=0x7fcd14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.371+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcd1c016030 con 0x7fcd241032f0 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:boot 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: fsmap 12 up:standby 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/334581286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:37 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/310875843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.585+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd241a39d0 con 0x7fcd241032f0 2024-04-26T21:09:37.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.586+0000 7fcd20c00640 1 -- 172.21.15.27:0/1430349363 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fcd1c09cfa0 con 0x7fcd241032f0 2024-04-26T21:09:37.587 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 msgr2=0x7fcd080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 0x7fcd080a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd24102790 tx=0x7fcd14005f90 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 msgr2=0x7fcd241a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00eaa0 tx=0x7fcd1c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 shutdown_connections 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcd080a3c30 0x7fcd080a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd241032f0 0x7fcd241a2170 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.588+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd24102930 0x7fcd2419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.589+0000 7fcd29800640 1 --2- 172.21.15.27:0/1430349363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd24101740 0x7fcd2419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.589+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 >> 172.21.15.27:0/1430349363 conn(0x7fcd240fccd0 msgr2=0x7fcd2410a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.589+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 shutdown_connections 2024-04-26T21:09:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.589+0000 7fcd29800640 1 -- 172.21.15.27:0/1430349363 wait complete. 2024-04-26T21:09:37.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:09:37.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:boot 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: fsmap 12 up:standby 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/334581286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:37 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/310875843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:boot 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: fsmap 12 up:standby 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/334581286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:37.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:37 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/310875843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 -- 172.21.15.27:0/4016283037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550103430 msgr2=0x7fe550105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 --2- 172.21.15.27:0/4016283037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550103430 0x7fe550105820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe540009900 tx=0x7fe54002a410 comp rx=0 tx=0).stop 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 -- 172.21.15.27:0/4016283037 shutdown_connections 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 --2- 172.21.15.27:0/4016283037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550108690 0x7fe55010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 --2- 172.21.15.27:0/4016283037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550105d60 0x7fe550108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 --2- 172.21.15.27:0/4016283037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550103430 0x7fe550105820 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.787+0000 7fe557200640 1 -- 172.21.15.27:0/4016283037 >> 172.21.15.27:0/4016283037 conn(0x7fe5500fce20 msgr2=0x7fe5500ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.788+0000 7fe557200640 1 -- 172.21.15.27:0/4016283037 shutdown_connections 2024-04-26T21:09:37.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.788+0000 7fe557200640 1 -- 172.21.15.27:0/4016283037 wait complete. 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.788+0000 7fe557200640 1 Processor -- start 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 -- start start 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550108690 0x7fe550101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010e450 con 0x7fe550108690 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010e2d0 con 0x7fe550103430 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe557200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010e5d0 con 0x7fe550105d60 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe54f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe54fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe54f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:41588/0 (socket says 172.21.15.27:41588) 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe54f400640 1 -- 172.21.15.27:0/4148971888 learned_addr learned my addr 172.21.15.27:0/4148971888 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe54fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:53614/0 (socket says 172.21.15.27:53614) 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.789+0000 7fe555200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550108690 0x7fe550101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 -- 172.21.15.27:0/4148971888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 msgr2=0x7fe550102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 -- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550108690 msgr2=0x7fe550101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550108690 0x7fe550101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 -- 172.21.15.27:0/4148971888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5501021b0 con 0x7fe550105d60 2024-04-26T21:09:37.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54fe00640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:09:37.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.790+0000 7fe54f400640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d770 tx=0x7fe53c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.791+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c017070 con 0x7fe550105d60 2024-04-26T21:09:37.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.791+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5501a2cf0 con 0x7fe550105d60 2024-04-26T21:09:37.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.791+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c009c30 con 0x7fe550105d60 2024-04-26T21:09:37.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.791+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c00f620 con 0x7fe550105d60 2024-04-26T21:09:37.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.791+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5501a3250 con 0x7fe550105d60 2024-04-26T21:09:37.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.792+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5500fe910 con 0x7fe550105d60 2024-04-26T21:09:37.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.795+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe53c01a070 con 0x7fe550105d60 2024-04-26T21:09:37.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.796+0000 7fe54cc00640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 0x7fe5340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.797+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe53c0cfa30 con 0x7fe550105d60 2024-04-26T21:09:37.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.797+0000 7fe54fe00640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 0x7fe5340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.798+0000 7fe54fe00640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 0x7fe5340a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe540005ab0 tx=0x7fe540005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:37.798+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe53c009de0 con 0x7fe550105d60 2024-04-26T21:09:38.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.004+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe55002dd20 con 0x7fe550105d60 2024-04-26T21:09:38.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.005+0000 7fe54cc00640 1 -- 172.21.15.27:0/4148971888 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7fe53c09c1c0 con 0x7fe550105d60 2024-04-26T21:09:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:09:38.007 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.008+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 msgr2=0x7fe5340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.008+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 0x7fe5340a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe540005ab0 tx=0x7fe540005a20 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.008+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 msgr2=0x7fe5501013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.008+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d770 tx=0x7fe53c00dc30 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 shutdown_connections 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe5340a3bb0 0x7fe5340a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe550105d60 0x7fe5501013e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe550108690 0x7fe550101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 --2- 172.21.15.27:0/4148971888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe550103430 0x7fe550102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 >> 172.21.15.27:0/4148971888 conn(0x7fe5500fce20 msgr2=0x7fe550105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:38.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 shutdown_connections 2024-04-26T21:09:38.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.009+0000 7fe557200640 1 -- 172.21.15.27:0/4148971888 wait complete. 2024-04-26T21:09:38.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:09:38.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:09:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 -- 172.21.15.27:0/1652011274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950103b70 msgr2=0x7f2950103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 --2- 172.21.15.27:0/1652011274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950103b70 0x7f2950103ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293802a410 comp rx=0 tx=0).stop 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 -- 172.21.15.27:0/1652011274 shutdown_connections 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 --2- 172.21.15.27:0/1652011274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950104530 0x7f295010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 --2- 172.21.15.27:0/1652011274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950103b70 0x7f2950103ff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 --2- 172.21.15.27:0/1652011274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950102980 0x7f2950102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 -- 172.21.15.27:0/1652011274 >> 172.21.15.27:0/1652011274 conn(0x7f29500fdf10 msgr2=0x7f2950100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 -- 172.21.15.27:0/1652011274 shutdown_connections 2024-04-26T21:09:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.241+0000 7f2955a00640 1 -- 172.21.15.27:0/1652011274 wait complete. 2024-04-26T21:09:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.243+0000 7f2955a00640 1 Processor -- start 2024-04-26T21:09:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.243+0000 7f2955a00640 1 -- start start 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.243+0000 7f2955a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2955a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950103b70 0x7f295019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2955a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950104530 0x7f29501a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2955a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e3b0 con 0x7f2950102980 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2955a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e230 con 0x7f2950104530 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2955a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e530 con 0x7f2950103b70 2024-04-26T21:09:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f294ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f294ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46634/0 (socket says 172.21.15.27:46634) 2024-04-26T21:09:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f294ea00640 1 -- 172.21.15.27:0/3492919713 learned_addr learned my addr 172.21.15.27:0/3492919713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f2947e00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950103b70 0x7f295019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.244+0000 7f294f400640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950104530 0x7f29501a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 -- 172.21.15.27:0/3492919713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950103b70 msgr2=0x7f295019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950103b70 0x7f295019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 -- 172.21.15.27:0/3492919713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950104530 msgr2=0x7f29501a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950104530 0x7f29501a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 -- 172.21.15.27:0/3492919713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29501a2950 con 0x7f2950102980 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.245+0000 7f294ea00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f294000d770 tx=0x7f294000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.246+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940017070 con 0x7f2950102980 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.246+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2940009c30 con 0x7f2950102980 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.246+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940003c50 con 0x7f2950102980 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.246+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501a2c40 con 0x7f2950102980 2024-04-26T21:09:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.246+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501a3190 con 0x7f2950102980 2024-04-26T21:09:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.247+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29501a3360 con 0x7f2950102980 2024-04-26T21:09:38.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.249+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2940003df0 con 0x7f2950102980 2024-04-26T21:09:38.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.251+0000 7f2947400640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 0x7f29340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.251+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f29400cf970 con 0x7f2950102980 2024-04-26T21:09:38.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.251+0000 7f2947e00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 0x7f29340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.252+0000 7f2947e00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 0x7f29340a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2938005ab0 tx=0x7f2938005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:38.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.255+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2940009de0 con 0x7f2950102980 2024-04-26T21:09:38.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:38.470+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f29501a35d0 con 0x7f2950102980 2024-04-26T21:09:38.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:38 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1430349363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:09:38.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:38 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4148971888' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:09:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:38 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1430349363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:09:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:38 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4148971888' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:09:38.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:38 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1430349363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:09:38.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:38 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4148971888' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:09:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.352+0000 7f2947400640 1 -- 172.21.15.27:0/3492919713 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f294009c100 con 0x7f2950102980 2024-04-26T21:09:39.353 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-04-26T21:09:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.355+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 msgr2=0x7f29340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.355+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 0x7f29340a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2938005ab0 tx=0x7f2938005a20 comp rx=0 tx=0).stop 2024-04-26T21:09:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.355+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 msgr2=0x7f295019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.355+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f294000d770 tx=0x7f294000dc30 comp rx=0 tx=0).stop 2024-04-26T21:09:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 shutdown_connections 2024-04-26T21:09:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29340a3c30 0x7f29340a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2950103b70 0x7f295019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2950104530 0x7f29501a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 --2- 172.21.15.27:0/3492919713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2950102980 0x7f295019ad40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.356+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 >> 172.21.15.27:0/3492919713 conn(0x7f29500fdf10 msgr2=0x7f2950109340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.357+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 shutdown_connections 2024-04-26T21:09:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.357+0000 7f2955a00640 1 -- 172.21.15.27:0/3492919713 wait complete. 2024-04-26T21:09:39.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:09:39.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 -- 172.21.15.27:0/3944131345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 msgr2=0x7f5644104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 --2- 172.21.15.27:0/3944131345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f5644104e90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5634009900 tx=0x7f563402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 -- 172.21.15.27:0/3944131345 shutdown_connections 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 --2- 172.21.15.27:0/3944131345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644106620 0x7f564410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 --2- 172.21.15.27:0/3944131345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644105c60 0x7f56441060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 --2- 172.21.15.27:0/3944131345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f5644104e90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 -- 172.21.15.27:0/3944131345 >> 172.21.15.27:0/3944131345 conn(0x7f56440fffe0 msgr2=0x7f5644102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 -- 172.21.15.27:0/3944131345 shutdown_connections 2024-04-26T21:09:39.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.583+0000 7f5643200640 1 -- 172.21.15.27:0/3944131345 wait complete. 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.584+0000 7f5643200640 1 Processor -- start 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.584+0000 7f5643200640 1 -- start start 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f564419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644105c60 0x7f564419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5644110420 con 0x7f5644106620 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56441102a0 con 0x7f5644104a70 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5643200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56441105a0 con 0x7f5644105c60 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5642800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5642800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53184/0 (socket says 172.21.15.27:53184) 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5642800640 1 -- 172.21.15.27:0/1494290999 learned_addr learned my addr 172.21.15.27:0/1494290999 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5641e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f564419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.585+0000 7f5641400640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644105c60 0x7f564419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 -- 172.21.15.27:0/1494290999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644105c60 msgr2=0x7f564419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644105c60 0x7f564419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 -- 172.21.15.27:0/1494290999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 msgr2=0x7f564419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f564419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 -- 172.21.15.27:0/1494290999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441a2800 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f5642800640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f563c00eaa0 tx=0x7f563c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.586+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c00cdb0 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.587+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56441a2af0 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.587+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f563c005020 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.587+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c0116d0 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.587+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441a3040 con 0x7f5644106620 2024-04-26T21:09:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.589+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564410bed0 con 0x7f5644106620 2024-04-26T21:09:39.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.592+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f563c0051d0 con 0x7f5644106620 2024-04-26T21:09:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.594+0000 7f563aa00640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 0x7f56280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.594+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f563c0d01e0 con 0x7f5644106620 2024-04-26T21:09:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.594+0000 7f5641e00640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 0x7f56280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.595+0000 7f5641e00640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 0x7f56280a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56340057c0 tx=0x7f5634031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:39.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.600+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f563c016030 con 0x7f5644106620 2024-04-26T21:09:39.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:39 smithi027 ceph-mon[28085]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:39.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:39 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:09:39.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:39 smithi204 ceph-mon[32145]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:39 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:09:39.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:39 smithi094 ceph-mon[31273]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:39.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:39 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:09:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:39.817+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f564402dad0 con 0x7f5644106620 2024-04-26T21:09:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.359+0000 7f563aa00640 1 -- 172.21.15.27:0/1494290999 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f563c09c840 con 0x7f5644106620 2024-04-26T21:09:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.362+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 msgr2=0x7f56280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.362+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 0x7f56280a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56340057c0 tx=0x7f5634031400 comp rx=0 tx=0).stop 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.362+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 msgr2=0x7f56441a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.362+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f563c00eaa0 tx=0x7f563c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 shutdown_connections 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56280a3c30 0x7f56280a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5644105c60 0x7f564419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644106620 0x7f56441a2190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 --2- 172.21.15.27:0/1494290999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5644104a70 0x7f564419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 >> 172.21.15.27:0/1494290999 conn(0x7f56440fffe0 msgr2=0x7f56441090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 shutdown_connections 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.363+0000 7f5643200640 1 -- 172.21.15.27:0/1494290999 wait complete. 2024-04-26T21:09:40.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 -- 172.21.15.27:0/1116503921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748106760 msgr2=0x7f874810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 --2- 172.21.15.27:0/1116503921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748106760 0x7f874810cff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8740009ca0 tx=0x7f874002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 -- 172.21.15.27:0/1116503921 shutdown_connections 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 --2- 172.21.15.27:0/1116503921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748106760 0x7f874810cff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 --2- 172.21.15.27:0/1116503921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748105da0 0x7f8748106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 --2- 172.21.15.27:0/1116503921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 0x7f8748104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 -- 172.21.15.27:0/1116503921 >> 172.21.15.27:0/1116503921 conn(0x7f8748100120 msgr2=0x7f8748102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 -- 172.21.15.27:0/1116503921 shutdown_connections 2024-04-26T21:09:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.599+0000 7f874fc00640 1 -- 172.21.15.27:0/1116503921 wait complete. 2024-04-26T21:09:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.601+0000 7f874fc00640 1 Processor -- start 2024-04-26T21:09:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.601+0000 7f874fc00640 1 -- start start 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 0x7f874819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 0x7f87481a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87481105e0 con 0x7f8748105da0 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8748110460 con 0x7f8748106760 2024-04-26T21:09:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8748110760 con 0x7f8748104bb0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f8747e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f8747e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53190/0 (socket says 172.21.15.27:53190) 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f8747e00640 1 -- 172.21.15.27:0/1342484572 learned_addr learned my addr 172.21.15.27:0/1342484572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874d200640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 0x7f874819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.602+0000 7f874dc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 0x7f87481a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 -- 172.21.15.27:0/1342484572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 msgr2=0x7f874819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 0x7f874819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 -- 172.21.15.27:0/1342484572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 msgr2=0x7f87481a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 0x7f87481a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 -- 172.21.15.27:0/1342484572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87481a2900 con 0x7f8748105da0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f874dc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 0x7f87481a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8747e00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f873400e9f0 tx=0x7f873400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400cd80 con 0x7f8748105da0 2024-04-26T21:09:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87340108c0 con 0x7f8748105da0 2024-04-26T21:09:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87481a2bf0 con 0x7f8748105da0 2024-04-26T21:09:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340187c0 con 0x7f8748105da0 2024-04-26T21:09:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.603+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87481a3190 con 0x7f8748105da0 2024-04-26T21:09:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.605+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874810bfa0 con 0x7f8748105da0 2024-04-26T21:09:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.607+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8734010430 con 0x7f8748105da0 2024-04-26T21:09:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.608+0000 7f8745600640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 0x7f87300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.608+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7f87340d0700 con 0x7f8748105da0 2024-04-26T21:09:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.609+0000 7f874d200640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 0x7f87300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.609+0000 7f874d200640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 0x7f87300a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f874819c1c0 tx=0x7f873c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.613+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f873409d1f0 con 0x7f8748105da0 2024-04-26T21:09:40.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:40 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:40 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:09:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3492919713' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:09:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:09:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:40.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:40 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:09:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:40.861+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f87480635a0 con 0x7f8748105da0 2024-04-26T21:09:41.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: pgmap v364: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:41.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:09:41.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:41.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:41 smithi204 ceph-mon[32145]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:09:41.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: pgmap v364: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:41 smithi027 ceph-mon[28085]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:09:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: pgmap v364: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:09:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1494290999' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:09:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:09:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:41.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:41 smithi094 ceph-mon[31273]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:09:41.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.777+0000 7f8745600640 1 -- 172.21.15.27:0/1342484572 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f873409caf0 con 0x7f8748105da0 2024-04-26T21:09:41.778 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:09:41.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.780+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 msgr2=0x7f87300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.780+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 0x7f87300a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f874819c1c0 tx=0x7f873c00b390 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.780+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 msgr2=0x7f874819b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.780+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f873400e9f0 tx=0x7f873400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 shutdown_connections 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f87300a3c30 0x7f87300a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8748106760 0x7f87481a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8748105da0 0x7f874819b230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 --2- 172.21.15.27:0/1342484572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8748104bb0 0x7f874819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 >> 172.21.15.27:0/1342484572 conn(0x7f8748100120 msgr2=0x7f8748109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 shutdown_connections 2024-04-26T21:09:41.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.781+0000 7f874fc00640 1 -- 172.21.15.27:0/1342484572 wait complete. 2024-04-26T21:09:41.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 -- 172.21.15.27:0/1240046140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e541039a0 msgr2=0x7f3e54103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 --2- 172.21.15.27:0/1240046140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e541039a0 0x7f3e54103e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02a410 comp rx=0 tx=0).stop 2024-04-26T21:09:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 -- 172.21.15.27:0/1240046140 shutdown_connections 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 --2- 172.21.15.27:0/1240046140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e54104360 0x7f3e5410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 --2- 172.21.15.27:0/1240046140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e541039a0 0x7f3e54103e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 --2- 172.21.15.27:0/1240046140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541027b0 0x7f3e54102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 -- 172.21.15.27:0/1240046140 >> 172.21.15.27:0/1240046140 conn(0x7f3e540fdd40 msgr2=0x7f3e54100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 -- 172.21.15.27:0/1240046140 shutdown_connections 2024-04-26T21:09:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:41.999+0000 7f3e53200640 1 -- 172.21.15.27:0/1240046140 wait complete. 2024-04-26T21:09:42.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.000+0000 7f3e53200640 1 Processor -- start 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.000+0000 7f3e53200640 1 -- start start 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e541027b0 0x7f3e5419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 0x7f3e5419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410e1c0 con 0x7f3e54104360 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410e040 con 0x7f3e541027b0 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e53200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410e340 con 0x7f3e541039a0 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e52800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e52800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53204/0 (socket says 172.21.15.27:53204) 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e52800640 1 -- 172.21.15.27:0/3522295462 learned_addr learned my addr 172.21.15.27:0/3522295462 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e51e00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e541027b0 0x7f3e5419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.001+0000 7f3e4be00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 0x7f3e5419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 -- 172.21.15.27:0/3522295462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 msgr2=0x7f3e5419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 0x7f3e5419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 -- 172.21.15.27:0/3522295462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e541027b0 msgr2=0x7f3e5419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e541027b0 0x7f3e5419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 -- 172.21.15.27:0/3522295462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e541a2610 con 0x7f3e54104360 2024-04-26T21:09:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e4be00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 0x7f3e5419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e52800640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00eaa0 tx=0x7f3e4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00cdb0 con 0x7f3e54104360 2024-04-26T21:09:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e4c005020 con 0x7f3e54104360 2024-04-26T21:09:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e541a2900 con 0x7f3e54104360 2024-04-26T21:09:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.002+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0116d0 con 0x7f3e54104360 2024-04-26T21:09:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.003+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e541a2dd0 con 0x7f3e54104360 2024-04-26T21:09:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.004+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e541a3030 con 0x7f3e54104360 2024-04-26T21:09:42.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.006+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e4c0051d0 con 0x7f3e54104360 2024-04-26T21:09:42.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.008+0000 7f3e4aa00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 0x7f3e380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.009+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3e4c0d0600 con 0x7f3e54104360 2024-04-26T21:09:42.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.009+0000 7f3e51e00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 0x7f3e380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.009+0000 7f3e51e00640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 0x7f3e380a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e440058c0 tx=0x7f3e44005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.014+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e4c016030 con 0x7f3e54104360 2024-04-26T21:09:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.243+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e541a32a0 con 0x7f3e54104360 2024-04-26T21:09:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.245+0000 7f3e4aa00640 1 -- 172.21.15.27:0/3522295462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f3e4c09d090 con 0x7f3e54104360 2024-04-26T21:09:42.246 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:42.247 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:09:42.028133+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 msgr2=0x7f3e380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 0x7f3e380a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e440058c0 tx=0x7f3e44005850 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 msgr2=0x7f3e541a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00eaa0 tx=0x7f3e4c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 shutdown_connections 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e380a3bb0 0x7f3e380a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e541039a0 0x7f3e5419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e54104360 0x7f3e541a1fa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 --2- 172.21.15.27:0/3522295462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e541027b0 0x7f3e5419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 >> 172.21.15.27:0/3522295462 conn(0x7f3e540fdd40 msgr2=0x7f3e54074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 shutdown_connections 2024-04-26T21:09:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.248+0000 7f3e53200640 1 -- 172.21.15.27:0/3522295462 wait complete. 2024-04-26T21:09:42.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 -- 172.21.15.27:0/3373161168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 msgr2=0x7fde7c10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 --2- 172.21.15.27:0/3373161168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c10ad90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fde74009ca0 tx=0x7fde7402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 -- 172.21.15.27:0/3373161168 shutdown_connections 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 --2- 172.21.15.27:0/3373161168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c10ad90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 --2- 172.21.15.27:0/3373161168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c103b40 0x7fde7c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 --2- 172.21.15.27:0/3373161168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c102950 0x7fde7c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.452+0000 7fde83000640 1 -- 172.21.15.27:0/3373161168 >> 172.21.15.27:0/3373161168 conn(0x7fde7c0fdee0 msgr2=0x7fde7c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.453+0000 7fde83000640 1 -- 172.21.15.27:0/3373161168 shutdown_connections 2024-04-26T21:09:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.453+0000 7fde83000640 1 -- 172.21.15.27:0/3373161168 wait complete. 2024-04-26T21:09:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 Processor -- start 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 -- start start 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c102950 0x7fde7c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 0x7fde7c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c10e360 con 0x7fde7c104500 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c10e1e0 con 0x7fde7c102950 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.454+0000 7fde83000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c10e4e0 con 0x7fde7c103b40 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde81000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 0x7fde7c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde81000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c1a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53228/0 (socket says 172.21.15.27:53228) 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde81000640 1 -- 172.21.15.27:0/4022617741 learned_addr learned my addr 172.21.15.27:0/4022617741 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7be00640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c102950 0x7fde7c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 -- 172.21.15.27:0/4022617741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c102950 msgr2=0x7fde7c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c102950 0x7fde7c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 -- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 msgr2=0x7fde7c1a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c1a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.455+0000 7fde7b400640 1 -- 172.21.15.27:0/4022617741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c1a0680 con 0x7fde7c103b40 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde7b400640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 0x7fde7c198fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9f0 tx=0x7fde6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6800ccc0 con 0x7fde7c103b40 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c074bc0 con 0x7fde7c103b40 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde680108c0 con 0x7fde7c103b40 2024-04-26T21:09:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680187c0 con 0x7fde7c103b40 2024-04-26T21:09:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.456+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c075110 con 0x7fde7c103b40 2024-04-26T21:09:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.458+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c109d40 con 0x7fde7c103b40 2024-04-26T21:09:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.460+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde68010430 con 0x7fde7c103b40 2024-04-26T21:09:42.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.462+0000 7fde78c00640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 0x7fde600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.462+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fde680d0600 con 0x7fde7c103b40 2024-04-26T21:09:42.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.462+0000 7fde7be00640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 0x7fde600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.463+0000 7fde7be00640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 0x7fde600a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde7c1039a0 tx=0x7fde6c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.471+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fde6809ca40 con 0x7fde7c103b40 2024-04-26T21:09:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.699+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde7c0764f0 con 0x7fde7c103b40 2024-04-26T21:09:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.704+0000 7fde78c00640 1 -- 172.21.15.27:0/4022617741 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7fde680a0d40 con 0x7fde7c103b40 2024-04-26T21:09:42.705 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:42.706 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:09:42.028133+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 msgr2=0x7fde600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 0x7fde600a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde7c1039a0 tx=0x7fde6c012040 comp rx=0 tx=0).stop 2024-04-26T21:09:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 msgr2=0x7fde7c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 0x7fde7c198fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9f0 tx=0x7fde6800eeb0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 shutdown_connections 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fde600a3bb0 0x7fde600a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde7c103b40 0x7fde7c198fa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde7c104500 0x7fde7c1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 --2- 172.21.15.27:0/4022617741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c102950 0x7fde7c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 >> 172.21.15.27:0/4022617741 conn(0x7fde7c0fdee0 msgr2=0x7fde7c106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 shutdown_connections 2024-04-26T21:09:42.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.707+0000 7fde83000640 1 -- 172.21.15.27:0/4022617741 wait complete. 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:09:42.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:42 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3522295462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:09:42.749 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:42 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3522295462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.750 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1342484572' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:09:42.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:09:42.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:09:42.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:09:42.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:42 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3522295462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 -- 172.21.15.27:0/2619315148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8076970 msgr2=0x7f5ea8074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 --2- 172.21.15.27:0/2619315148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8076970 0x7f5ea8074dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009cd0 tx=0x7f5ea002a8e0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 -- 172.21.15.27:0/2619315148 shutdown_connections 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 --2- 172.21.15.27:0/2619315148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea810ee90 0x7f5ea8111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 --2- 172.21.15.27:0/2619315148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea8075390 0x7f5ea8075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 --2- 172.21.15.27:0/2619315148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8076970 0x7f5ea8074dc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 -- 172.21.15.27:0/2619315148 >> 172.21.15.27:0/2619315148 conn(0x7f5ea80fde30 msgr2=0x7f5ea8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 -- 172.21.15.27:0/2619315148 shutdown_connections 2024-04-26T21:09:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.949+0000 7f5eace00640 1 -- 172.21.15.27:0/2619315148 wait complete. 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.950+0000 7f5eace00640 1 Processor -- start 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 -- start start 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea8076970 0x7f5ea819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea810ee90 0x7f5ea81a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8114850 con 0x7f5ea8075390 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea81146d0 con 0x7f5ea810ee90 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5eace00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea81149d0 con 0x7f5ea8076970 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5ea7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5ea7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53246/0 (socket says 172.21.15.27:53246) 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5ea7400640 1 -- 172.21.15.27:0/2197332336 learned_addr learned my addr 172.21.15.27:0/2197332336 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5ea6a00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea8076970 0x7f5ea819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.951+0000 7f5ea7e00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea810ee90 0x7f5ea81a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 -- 172.21.15.27:0/2197332336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea8076970 msgr2=0x7f5ea819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea8076970 0x7f5ea819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 -- 172.21.15.27:0/2197332336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea810ee90 msgr2=0x7f5ea81a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea810ee90 0x7f5ea81a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 -- 172.21.15.27:0/2197332336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea81a6c60 con 0x7f5ea8075390 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5ea7400640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0031730 tx=0x7f5ea0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0004240 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ea00305a0 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea81a6ef0 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00466e0 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.952+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a73b0 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.954+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea8105060 con 0x7f5ea8075390 2024-04-26T21:09:42.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.956+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ea0030a10 con 0x7f5ea8075390 2024-04-26T21:09:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.957+0000 7f5e9fe00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 0x7f5e8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.957+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5ea002e030 con 0x7f5ea8075390 2024-04-26T21:09:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.957+0000 7f5ea6a00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 0x7f5e8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.958+0000 7f5ea6a00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 0x7f5e8c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ea81a0550 tx=0x7f5e90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:42.962+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ea004b060 con 0x7f5ea8075390 2024-04-26T21:09:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:43.225+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5ea802dcf0 con 0x7f5ea8075390 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: pgmap v368: 97 pgs: 10 creating+peering, 30 active+clean, 57 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4022617741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:active 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: Cluster is now healthy 2024-04-26T21:09:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:43 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: pgmap v368: 97 pgs: 10 creating+peering, 30 active+clean, 57 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4022617741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:active 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: Cluster is now healthy 2024-04-26T21:09:43.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:43 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:09:43.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: pgmap v368: 97 pgs: 10 creating+peering, 30 active+clean, 57 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4022617741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] up:active 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: Cluster is now healthy 2024-04-26T21:09:43.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:43 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:09:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.037+0000 7f5e9fe00640 1 -- 172.21.15.27:0/2197332336 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f5ea00be490 con 0x7f5ea8075390 2024-04-26T21:09:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 msgr2=0x7f5e8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 0x7f5e8c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ea81a0550 tx=0x7f5e90005f90 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 msgr2=0x7f5ea819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0031730 tx=0x7f5ea0003ba0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 shutdown_connections 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f5e8c0a3c30 0x7f5e8c0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ea8076970 0x7f5ea819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ea810ee90 0x7f5ea81a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 --2- 172.21.15.27:0/2197332336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea8075390 0x7f5ea819f040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 >> 172.21.15.27:0/2197332336 conn(0x7f5ea80fde30 msgr2=0x7f5ea8102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 shutdown_connections 2024-04-26T21:09:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.041+0000 7f5eace00640 1 -- 172.21.15.27:0/2197332336 wait complete. 2024-04-26T21:09:44.063 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T21:09:44.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 -- 172.21.15.27:0/1515233871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078530 msgr2=0x7f6698078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 --2- 172.21.15.27:0/1515233871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078530 0x7f6698078990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6684009900 tx=0x7f668402a410 comp rx=0 tx=0).stop 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 -- 172.21.15.27:0/1515233871 shutdown_connections 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 --2- 172.21.15.27:0/1515233871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078ed0 0x7f6698079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 --2- 172.21.15.27:0/1515233871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078530 0x7f6698078990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 --2- 172.21.15.27:0/1515233871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 0x7f6698077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 -- 172.21.15.27:0/1515233871 >> 172.21.15.27:0/1515233871 conn(0x7f66981000e0 msgr2=0x7f6698102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 -- 172.21.15.27:0/1515233871 shutdown_connections 2024-04-26T21:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.269+0000 7f669d000640 1 -- 172.21.15.27:0/1515233871 wait complete. 2024-04-26T21:09:44.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.270+0000 7f669d000640 1 Processor -- start 2024-04-26T21:09:44.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.270+0000 7f669d000640 1 -- start start 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 0x7f669819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078530 0x7f669819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6698114980 con 0x7f6698078ed0 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6698114800 con 0x7f66980772f0 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f669d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6698114b00 con 0x7f6698078530 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f6696800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f6696800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53278/0 (socket says 172.21.15.27:53278) 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f6696800640 1 -- 172.21.15.27:0/3836633666 learned_addr learned my addr 172.21.15.27:0/3836633666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f6695e00640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 0x7f669819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.271+0000 7f6695400640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078530 0x7f669819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 -- 172.21.15.27:0/3836633666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078530 msgr2=0x7f669819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078530 0x7f669819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 -- 172.21.15.27:0/3836633666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 msgr2=0x7f669819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 0x7f669819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 -- 172.21.15.27:0/3836633666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66981a6d20 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f6696800640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f669000eb10 tx=0x7f669000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690004b20 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6690004cc0 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66981a6fb0 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.272+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690010690 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.273+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a7520 con 0x7f6698078ed0 2024-04-26T21:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.274+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66981117e0 con 0x7f6698078ed0 2024-04-26T21:09:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.276+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6690005020 con 0x7f6698078ed0 2024-04-26T21:09:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.278+0000 7f668ea00640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 0x7f667c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.279+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f66900d08f0 con 0x7f6698078ed0 2024-04-26T21:09:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.279+0000 7f6695e00640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 0x7f667c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.279+0000 7f6695e00640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 0x7f667c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6688005d00 tx=0x7f6688005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.283+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f669009d380 con 0x7f6698078ed0 2024-04-26T21:09:44.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:44.556+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6698063530 con 0x7f6698078ed0 2024-04-26T21:09:45.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: pgmap v369: 97 pgs: 10 creating+peering, 76 active+clean, 11 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:45 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: pgmap v369: 97 pgs: 10 creating+peering, 76 active+clean, 11 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:45 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2197332336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: pgmap v369: 97 pgs: 10 creating+peering, 76 active+clean, 11 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:45.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:45.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:09:45.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:45 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:09:45.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.533+0000 7f668ea00640 1 -- 172.21.15.27:0/3836633666 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f669009cc80 con 0x7f6698078ed0 2024-04-26T21:09:45.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.537+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 msgr2=0x7f667c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.537+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 0x7f667c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6688005d00 tx=0x7f6688005c50 comp rx=0 tx=0).stop 2024-04-26T21:09:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.537+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 msgr2=0x7f66981a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.537+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f669000eb10 tx=0x7f669000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 shutdown_connections 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f667c0a3c30 0x7f667c0a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6698078530 0x7f669819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6698078ed0 0x7f66981a66b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 --2- 172.21.15.27:0/3836633666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66980772f0 0x7f669819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 >> 172.21.15.27:0/3836633666 conn(0x7f66981000e0 msgr2=0x7f6698101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 shutdown_connections 2024-04-26T21:09:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.538+0000 7f669d000640 1 -- 172.21.15.27:0/3836633666 wait complete. 2024-04-26T21:09:45.562 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 -- 172.21.15.27:0/1853445012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64103a50 msgr2=0x7f2e64103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 --2- 172.21.15.27:0/1853445012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64103a50 0x7f2e64103ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009900 tx=0x7f2e5402a410 comp rx=0 tx=0).stop 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 -- 172.21.15.27:0/1853445012 shutdown_connections 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 --2- 172.21.15.27:0/1853445012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64104410 0x7f2e6410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 --2- 172.21.15.27:0/1853445012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64103a50 0x7f2e64103ed0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 --2- 172.21.15.27:0/1853445012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64102860 0x7f2e64102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 -- 172.21.15.27:0/1853445012 >> 172.21.15.27:0/1853445012 conn(0x7f2e640fddd0 msgr2=0x7f2e64100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 -- 172.21.15.27:0/1853445012 shutdown_connections 2024-04-26T21:09:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.769+0000 7f2e6b200640 1 -- 172.21.15.27:0/1853445012 wait complete. 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 Processor -- start 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 -- start start 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64103a50 0x7f2e64074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 0x7f2e64075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410e290 con 0x7f2e64102860 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410e110 con 0x7f2e64104410 2024-04-26T21:09:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.771+0000 7f2e6b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410e410 con 0x7f2e64103a50 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53306/0 (socket says 172.21.15.27:53306) 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 -- 172.21.15.27:0/2568865608 learned_addr learned my addr 172.21.15.27:0/2568865608 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69400640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64103a50 0x7f2e64074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e6a800640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 0x7f2e64075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 -- 172.21.15.27:0/2568865608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64103a50 msgr2=0x7f2e64074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64103a50 0x7f2e64074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 -- 172.21.15.27:0/2568865608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 msgr2=0x7f2e64075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 0x7f2e64075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e69e00640 1 -- 172.21.15.27:0/2568865608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e64075af0 con 0x7f2e64102860 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.772+0000 7f2e6a800640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 0x7f2e64075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e69e00640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00eaa0 tx=0x7f2e5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c016070 con 0x7f2e64102860 2024-04-26T21:09:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e5c00ce10 con 0x7f2e64102860 2024-04-26T21:09:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641a2ab0 con 0x7f2e64102860 2024-04-26T21:09:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c011470 con 0x7f2e64102860 2024-04-26T21:09:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.773+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641a2fd0 con 0x7f2e64102860 2024-04-26T21:09:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.774+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e64069670 con 0x7f2e64102860 2024-04-26T21:09:45.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.777+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e5c019070 con 0x7f2e64102860 2024-04-26T21:09:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.778+0000 7f2e5aa00640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 0x7f2e500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.778+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2e5c0d0450 con 0x7f2e64102860 2024-04-26T21:09:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.778+0000 7f2e69400640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 0x7f2e500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.779+0000 7f2e69400640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 0x7f2e500a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006010 tx=0x7f2e54031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:45.783+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e5c09cee0 con 0x7f2e64102860 2024-04-26T21:09:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.035+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2e64109c20 con 0x7f2e64102860 2024-04-26T21:09:46.356 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:09:46.357 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:09:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:46 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3836633666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:09:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:46 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:09:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.549+0000 7f2e5aa00640 1 -- 172.21.15.27:0/2568865608 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f2e64109c20 con 0x7f2e64102860 2024-04-26T21:09:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.552+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 msgr2=0x7f2e500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.552+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 0x7f2e500a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006010 tx=0x7f2e54031400 comp rx=0 tx=0).stop 2024-04-26T21:09:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.552+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 msgr2=0x7f2e640766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.552+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00eaa0 tx=0x7f2e5c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.552+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 shutdown_connections 2024-04-26T21:09:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2e500a3c30 0x7f2e500a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e64103a50 0x7f2e64074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e64104410 0x7f2e64075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 --2- 172.21.15.27:0/2568865608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e64102860 0x7f2e640766e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 >> 172.21.15.27:0/2568865608 conn(0x7f2e640fddd0 msgr2=0x7f2e64106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 shutdown_connections 2024-04-26T21:09:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.553+0000 7f2e6b200640 1 -- 172.21.15.27:0/2568865608 wait complete. 2024-04-26T21:09:46.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T21:09:46.776 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: 2024-04-26T21:09:46.354+0000 7f28fc400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:09:46.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 -- 172.21.15.27:0/1058361794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174101110 msgr2=0x7ff174101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 --2- 172.21.15.27:0/1058361794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174101110 0x7ff174101530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff164009900 tx=0x7ff16402a410 comp rx=0 tx=0).stop 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 -- 172.21.15.27:0/1058361794 shutdown_connections 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 --2- 172.21.15.27:0/1058361794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 0x7ff174101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 --2- 172.21.15.27:0/1058361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174102410 0x7ff174102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 --2- 172.21.15.27:0/1058361794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174101110 0x7ff174101530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 -- 172.21.15.27:0/1058361794 >> 172.21.15.27:0/1058361794 conn(0x7ff1740fccd0 msgr2=0x7ff1740ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 -- 172.21.15.27:0/1058361794 shutdown_connections 2024-04-26T21:09:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.777+0000 7ff173200640 1 -- 172.21.15.27:0/1058361794 wait complete. 2024-04-26T21:09:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.778+0000 7ff173200640 1 Processor -- start 2024-04-26T21:09:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.778+0000 7ff173200640 1 -- start start 2024-04-26T21:09:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174101110 0x7ff17419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 0x7ff17419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff174110450 con 0x7ff174102410 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1741102d0 con 0x7ff174101110 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff173200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1741105d0 con 0x7ff174101a70 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff172800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff172800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53316/0 (socket says 172.21.15.27:53316) 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff172800640 1 -- 172.21.15.27:0/1666036875 learned_addr learned my addr 172.21.15.27:0/1666036875 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff171e00640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174101110 0x7ff17419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff171400640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 0x7ff17419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff172800640 1 -- 172.21.15.27:0/1666036875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 msgr2=0x7ff17419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.779+0000 7ff172800640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 0x7ff17419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff172800640 1 -- 172.21.15.27:0/1666036875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174101110 msgr2=0x7ff17419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff172800640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174101110 0x7ff17419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff172800640 1 -- 172.21.15.27:0/1666036875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1741a27d0 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff172800640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00cad0 tx=0x7ff16c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00ae20 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff16c005020 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1741a2ac0 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00f6b0 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.780+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741a2fc0 con 0x7ff174102410 2024-04-26T21:09:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.781+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17410c140 con 0x7ff174102410 2024-04-26T21:09:46.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.783+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff16c0051d0 con 0x7ff174102410 2024-04-26T21:09:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.785+0000 7ff16aa00640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 0x7ff1540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.785+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff16c0d07c0 con 0x7ff174102410 2024-04-26T21:09:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.785+0000 7ff171e00640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 0x7ff1540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.786+0000 7ff171e00640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 0x7ff1540a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff164031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:46.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:46.790+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff16c014030 con 0x7ff174102410 2024-04-26T21:09:47.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.038+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff17402dad0 con 0x7ff174102410 2024-04-26T21:09:47.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.564+0000 7ff16aa00640 1 -- 172.21.15.27:0/1666036875 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7ff16c09d250 con 0x7ff174102410 2024-04-26T21:09:47.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.568+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 msgr2=0x7ff1540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.568+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 0x7ff1540a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff164031400 comp rx=0 tx=0).stop 2024-04-26T21:09:47.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.568+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 msgr2=0x7ff1741a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.568+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00cad0 tx=0x7ff16c00cf90 comp rx=0 tx=0).stop 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 shutdown_connections 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff1540a3bb0 0x7ff1540a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff174101a70 0x7ff17419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff174102410 0x7ff1741a2160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 --2- 172.21.15.27:0/1666036875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff174101110 0x7ff17419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 >> 172.21.15.27:0/1666036875 conn(0x7ff1740fccd0 msgr2=0x7ff1740fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 shutdown_connections 2024-04-26T21:09:47.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.569+0000 7ff173200640 1 -- 172.21.15.27:0/1666036875 wait complete. 2024-04-26T21:09:47.598 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T21:09:47.599 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T21:09:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.796+0000 7fab76200640 1 -- 172.21.15.27:0/3985834919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 msgr2=0x7fab7010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.796+0000 7fab76200640 1 --2- 172.21.15.27:0/3985834919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7010a440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009900 tx=0x7fab5c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 -- 172.21.15.27:0/3985834919 shutdown_connections 2024-04-26T21:09:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 --2- 172.21.15.27:0/3985834919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab7010a980 0x7fab701112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 --2- 172.21.15.27:0/3985834919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7010a440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 --2- 172.21.15.27:0/3985834919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab70108e70 0x7fab70109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 -- 172.21.15.27:0/3985834919 >> 172.21.15.27:0/3985834919 conn(0x7fab70077e00 msgr2=0x7fab7007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 -- 172.21.15.27:0/3985834919 shutdown_connections 2024-04-26T21:09:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.797+0000 7fab76200640 1 -- 172.21.15.27:0/3985834919 wait complete. 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 Processor -- start 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 -- start start 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70108e70 0x7fab7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7010a980 0x7fab701a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab70114850 con 0x7fab70109fc0 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab701146d0 con 0x7fab70108e70 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.798+0000 7fab76200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab701149d0 con 0x7fab7010a980 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53342/0 (socket says 172.21.15.27:53342) 2024-04-26T21:09:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 -- 172.21.15.27:0/4124013004 learned_addr learned my addr 172.21.15.27:0/4124013004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab74e00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70108e70 0x7fab7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab75800640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7010a980 0x7fab701a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 -- 172.21.15.27:0/4124013004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7010a980 msgr2=0x7fab701a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7010a980 0x7fab701a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 -- 172.21.15.27:0/4124013004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70108e70 msgr2=0x7fab7019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70108e70 0x7fab7019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 -- 172.21.15.27:0/4124013004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab701a6b20 con 0x7fab70109fc0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.799+0000 7fab6fe00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009230 tx=0x7fab5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.800+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c03f030 con 0x7fab70109fc0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.800+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab5c0044c0 con 0x7fab70109fc0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.800+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab701a6db0 con 0x7fab70109fc0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.800+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c035070 con 0x7fab70109fc0 2024-04-26T21:09:47.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.800+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab701a71e0 con 0x7fab70109fc0 2024-04-26T21:09:47.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.801+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab70110180 con 0x7fab70109fc0 2024-04-26T21:09:47.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.803+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab5c042030 con 0x7fab70109fc0 2024-04-26T21:09:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.805+0000 7fab6d600640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.805+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fab5c0f0800 con 0x7fab70109fc0 2024-04-26T21:09:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.805+0000 7fab74e00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.806+0000 7fab74e00640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 0x7fab580a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab6400abe0 tx=0x7fab6400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:47.810+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab5c0bd330 con 0x7fab70109fc0 2024-04-26T21:09:47.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: pgmap v370: 97 pgs: 97 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:active 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:09:47.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:47 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:09:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: pgmap v370: 97 pgs: 97 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-26T21:09:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:09:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:active 2024-04-26T21:09:48.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:09:48.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:09:48.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:09:48.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:09:48.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:47 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: pgmap v370: 97 pgs: 97 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2568865608' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] up:active 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:09:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:47 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:09:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.045+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fab70062d70 con 0x7fab580a3bb0 2024-04-26T21:09:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.102+0000 7fab6d600640 1 -- 172.21.15.27:0/4124013004 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab70062d70 con 0x7fab580a3bb0 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 msgr2=0x7fab580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 0x7fab580a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab6400abe0 tx=0x7fab6400a2d0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 msgr2=0x7fab7019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009230 tx=0x7fab5c0040c0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 shutdown_connections 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7010a980 0x7fab701a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab70109fc0 0x7fab7019f440 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 --2- 172.21.15.27:0/4124013004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70108e70 0x7fab7019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.105+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 >> 172.21.15.27:0/4124013004 conn(0x7fab70077e00 msgr2=0x7fab7010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.106+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 shutdown_connections 2024-04-26T21:09:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.106+0000 7fab76200640 1 -- 172.21.15.27:0/4124013004 wait complete. 2024-04-26T21:09:48.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 -- 172.21.15.27:0/482600586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07810ef50 msgr2=0x7fc078111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 --2- 172.21.15.27:0/482600586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07810ef50 0x7fc078111340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc070009cd0 tx=0x7fc07002a8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 -- 172.21.15.27:0/482600586 shutdown_connections 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 --2- 172.21.15.27:0/482600586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc078075390 0x7fc078075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 --2- 172.21.15.27:0/482600586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07810ef50 0x7fc078111340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 --2- 172.21.15.27:0/482600586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc078076970 0x7fc078074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 -- 172.21.15.27:0/482600586 >> 172.21.15.27:0/482600586 conn(0x7fc0780fdf10 msgr2=0x7fc078100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.344+0000 7fc07e600640 1 -- 172.21.15.27:0/482600586 shutdown_connections 2024-04-26T21:09:48.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.345+0000 7fc07e600640 1 -- 172.21.15.27:0/482600586 wait complete. 2024-04-26T21:09:48.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.345+0000 7fc07e600640 1 Processor -- start 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 -- start start 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc078076970 0x7fc07819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc07810ef50 0x7fc0781a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc078114980 con 0x7fc078075390 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc078114800 con 0x7fc07810ef50 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc07e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc078114b00 con 0x7fc078076970 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc077400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc077400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53372/0 (socket says 172.21.15.27:53372) 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.346+0000 7fc077400640 1 -- 172.21.15.27:0/1498014442 learned_addr learned my addr 172.21.15.27:0/1498014442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077e00640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc07810ef50 0x7fc0781a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 -- 172.21.15.27:0/1498014442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc078076970 msgr2=0x7fc07819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc078076970 0x7fc07819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 -- 172.21.15.27:0/1498014442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc07810ef50 msgr2=0x7fc0781a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc07810ef50 0x7fc0781a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 -- 172.21.15.27:0/1498014442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0781a6cd0 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc077400640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e9f0 tx=0x7fc06800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800cd80 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0680108c0 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680187c0 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.347+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0781a6fc0 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.348+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a7430 con 0x7fc078075390 2024-04-26T21:09:48.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.349+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078105120 con 0x7fc078075390 2024-04-26T21:09:48.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.351+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc068010430 con 0x7fc078075390 2024-04-26T21:09:48.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.352+0000 7fc06fe00640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 0x7fc05c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.352+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc0680d0750 con 0x7fc078075390 2024-04-26T21:09:48.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.353+0000 7fc076a00640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 0x7fc05c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.353+0000 7fc076a00640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 0x7fc05c0a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fd0 tx=0x7fc064005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.356+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc06809d1e0 con 0x7fc078075390 2024-04-26T21:09:48.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:09:48.053+0000 7f070a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:09:48.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:09:48.053+0000 7f070a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:09:48.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:09:48.053+0000 7f070a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:09:48.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:09:48.053+0000 7f070a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:09:48.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:09:48.053+0000 7f070a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:09:48.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.616+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc068099150 con 0x7fc078075390 2024-04-26T21:09:48.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.620+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc078062e00 con 0x7fc05c0a3c30 2024-04-26T21:09:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.681+0000 7fc06fe00640 1 -- 172.21.15.27:0/1498014442 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc078062e00 con 0x7fc05c0a3c30 2024-04-26T21:09:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.684+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 msgr2=0x7fc05c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.684+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 0x7fc05c0a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fd0 tx=0x7fc064005850 comp rx=0 tx=0).stop 2024-04-26T21:09:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.684+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 msgr2=0x7fc07819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.684+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e9f0 tx=0x7fc06800eeb0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 shutdown_connections 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc05c0a3c30 0x7fc05c0a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc078076970 0x7fc07819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc07810ef50 0x7fc0781a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 --2- 172.21.15.27:0/1498014442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc078075390 0x7fc07819f0b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 >> 172.21.15.27:0/1498014442 conn(0x7fc0780fdf10 msgr2=0x7fc0781022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 shutdown_connections 2024-04-26T21:09:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.685+0000 7fc07e600640 1 -- 172.21.15.27:0/1498014442 wait complete. 2024-04-26T21:09:48.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T21:09:48.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 -- 172.21.15.27:0/1969114057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08102860 msgr2=0x7f0c08102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 --2- 172.21.15.27:0/1969114057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08102860 0x7f0c08102c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009ca0 tx=0x7f0bf802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 -- 172.21.15.27:0/1969114057 shutdown_connections 2024-04-26T21:09:48.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 --2- 172.21.15.27:0/1969114057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08104410 0x7f0c0810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 --2- 172.21.15.27:0/1969114057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08103a50 0x7f0c08103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 --2- 172.21.15.27:0/1969114057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08102860 0x7f0c08102c80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 -- 172.21.15.27:0/1969114057 >> 172.21.15.27:0/1969114057 conn(0x7f0c080fddd0 msgr2=0x7f0c08100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 -- 172.21.15.27:0/1969114057 shutdown_connections 2024-04-26T21:09:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.923+0000 7f0c0d600640 1 -- 172.21.15.27:0/1969114057 wait complete. 2024-04-26T21:09:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.924+0000 7f0c0d600640 1 Processor -- start 2024-04-26T21:09:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.924+0000 7f0c0d600640 1 -- start start 2024-04-26T21:09:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.924+0000 7f0c0d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 0x7f0c0819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.924+0000 7f0c0d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08103a50 0x7f0c0819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0d600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810e290 con 0x7f0c08104410 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810e110 con 0x7f0c08102860 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810e410 con 0x7f0c08103a50 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53392/0 (socket says 172.21.15.27:53392) 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 -- 172.21.15.27:0/4224193500 learned_addr learned my addr 172.21.15.27:0/4224193500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c07e00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 0x7f0c0819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c07400640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08103a50 0x7f0c0819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 -- 172.21.15.27:0/4224193500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08103a50 msgr2=0x7f0c0819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08103a50 0x7f0c0819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 -- 172.21.15.27:0/4224193500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 msgr2=0x7f0c0819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 0x7f0c0819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c0cc00640 1 -- 172.21.15.27:0/4224193500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c081a2830 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.925+0000 7f0c07e00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 0x7f0c0819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c0cc00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000d750 tx=0x7f0c0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00017070 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c081a2b20 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c000054b0 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004b90 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.926+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a3070 con 0x7f0c08104410 2024-04-26T21:09:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.927+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c081a3240 con 0x7f0c08104410 2024-04-26T21:09:48.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.930+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c00004d30 con 0x7f0c08104410 2024-04-26T21:09:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.932+0000 7f0c04c00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.932+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0c000cfff0 con 0x7f0c08104410 2024-04-26T21:09:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.932+0000 7f0c07e00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.933+0000 7f0c07e00640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 0x7f0bec0a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006010 tx=0x7f0bf8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:48.936+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c0009c960 con 0x7f0c08104410 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:active 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:active 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: Cluster is now healthy 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-26T21:09:48.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:48 smithi027 ceph-mon[28085]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-26T21:09:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:09:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:active 2024-04-26T21:09:49.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:active 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: Cluster is now healthy 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-26T21:09:49.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:48 smithi204 ceph-mon[32145]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1666036875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] up:active 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:09:49.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] up:active 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: Cluster is now healthy 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-26T21:09:49.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:48 smithi094 ceph-mon[31273]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-26T21:09:49.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.185+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0c0810a9e0 con 0x7f0bec0a3c40 2024-04-26T21:09:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.198+0000 7f0c04c00640 1 -- 172.21.15.27:0/4224193500 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0c0810a9e0 con 0x7f0bec0a3c40 2024-04-26T21:09:49.199 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/4f65e73b-e35b-4043-a5ec-4c24d8b66b8e 2024-04-26T21:09:49.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 msgr2=0x7f0bec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 0x7f0bec0a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006010 tx=0x7f0bf8009360 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 msgr2=0x7f0c081a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000d750 tx=0x7f0c0000dc10 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 shutdown_connections 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c08103a50 0x7f0c0819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c08104410 0x7f0c081a2190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 --2- 172.21.15.27:0/4224193500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c08102860 0x7f0c0819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 >> 172.21.15.27:0/4224193500 conn(0x7f0c080fddd0 msgr2=0x7f0c081092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.200+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 shutdown_connections 2024-04-26T21:09:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.201+0000 7f0c0d600640 1 -- 172.21.15.27:0/4224193500 wait complete. 2024-04-26T21:09:49.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 -- 172.21.15.27:0/1263145134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d8108e70 msgr2=0x7f95d8109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 --2- 172.21.15.27:0/1263145134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d8108e70 0x7f95d8109290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009900 tx=0x7f95c802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 -- 172.21.15.27:0/1263145134 shutdown_connections 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 --2- 172.21.15.27:0/1263145134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 0x7f95d810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 --2- 172.21.15.27:0/1263145134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d810a980 0x7f95d81112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 --2- 172.21.15.27:0/1263145134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d8108e70 0x7f95d8109290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 -- 172.21.15.27:0/1263145134 >> 172.21.15.27:0/1263145134 conn(0x7f95d8077e00 msgr2=0x7f95d807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.427+0000 7f95dca00640 1 -- 172.21.15.27:0/1263145134 shutdown_connections 2024-04-26T21:09:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.428+0000 7f95dca00640 1 -- 172.21.15.27:0/1263145134 wait complete. 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.428+0000 7f95dca00640 1 Processor -- start 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 -- start start 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d8108e70 0x7f95d819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 0x7f95d819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81148a0 con 0x7f95d810a980 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8114720 con 0x7f95d8108e70 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95dca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8114a20 con 0x7f95d8109fc0 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95d7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95d7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36272/0 (socket says 172.21.15.27:36272) 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.429+0000 7f95d7200640 1 -- 172.21.15.27:0/1478204165 learned_addr learned my addr 172.21.15.27:0/1478204165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d6800640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d8108e70 0x7f95d819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95cde00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 0x7f95d819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 -- 172.21.15.27:0/1478204165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 msgr2=0x7f95d819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 0x7f95d819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 -- 172.21.15.27:0/1478204165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d8108e70 msgr2=0x7f95d819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d8108e70 0x7f95d819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 -- 172.21.15.27:0/1478204165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81a6b30 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95d7200640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eaa0 tx=0x7f95d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.430+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000cdb0 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.431+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95d0005020 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.431+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81a6e20 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.431+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00116d0 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.431+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a7290 con 0x7f95d810a980 2024-04-26T21:09:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.432+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d8110180 con 0x7f95d810a980 2024-04-26T21:09:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.434+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f95d00051d0 con 0x7f95d810a980 2024-04-26T21:09:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.436+0000 7f95cfe00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 0x7f95bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.437+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f95d00d0a00 con 0x7f95d810a980 2024-04-26T21:09:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.437+0000 7f95d6800640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 0x7f95bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.438+0000 7f95d6800640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 0x7f95bc0a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005ab0 tx=0x7f95c8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.441+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f95d0016030 con 0x7f95d810a980 2024-04-26T21:09:49.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.691+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f95d8062d70 con 0x7f95bc0a3c40 2024-04-26T21:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.766+0000 7f95cfe00640 1 -- 172.21.15.27:0/1478204165 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95d8062d70 con 0x7f95bc0a3c40 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 msgr2=0x7f95bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 0x7f95bc0a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005ab0 tx=0x7f95c8005a20 comp rx=0 tx=0).stop 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 msgr2=0x7f95d81a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eaa0 tx=0x7f95d000ef60 comp rx=0 tx=0).stop 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 shutdown_connections 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95bc0a3c40 0x7f95bc0a6100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95d8109fc0 0x7f95d819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d810a980 0x7f95d81a64c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 --2- 172.21.15.27:0/1478204165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95d8108e70 0x7f95d819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.769+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 >> 172.21.15.27:0/1478204165 conn(0x7f95d8077e00 msgr2=0x7f95d810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.770+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 shutdown_connections 2024-04-26T21:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:49.770+0000 7f95dca00640 1 -- 172.21.15.27:0/1478204165 wait complete. 2024-04-26T21:09:49.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: pgmap v371: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 11 op/s 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:active 2024-04-26T21:09:49.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-26T21:09:49.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:49 smithi027 ceph-mon[28085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:09:50.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: pgmap v371: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 11 op/s 2024-04-26T21:09:50.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:50.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:09:50.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:50.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:active 2024-04-26T21:09:50.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-26T21:09:50.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:49 smithi204 ceph-mon[32145]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:09:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 -- 172.21.15.27:0/2678367208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf410ab40 msgr2=0x7f2cf41113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 --2- 172.21.15.27:0/2678367208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf410ab40 0x7f2cf41113d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009ca0 tx=0x7f2cec02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 -- 172.21.15.27:0/2678367208 shutdown_connections 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 --2- 172.21.15.27:0/2678367208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 --2- 172.21.15.27:0/2678367208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf410ab40 0x7f2cf41113d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 --2- 172.21.15.27:0/2678367208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf4108f90 0x7f2cf41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 -- 172.21.15.27:0/2678367208 >> 172.21.15.27:0/2678367208 conn(0x7f2cf4077f30 msgr2=0x7f2cf407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.011+0000 7f2cfba00640 1 -- 172.21.15.27:0/2678367208 shutdown_connections 2024-04-26T21:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.012+0000 7f2cfba00640 1 -- 172.21.15.27:0/2678367208 wait complete. 2024-04-26T21:09:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.012+0000 7f2cfba00640 1 Processor -- start 2024-04-26T21:09:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 -- start start 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf410ab40 0x7f2cf41a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41149d0 con 0x7f2cf4108f90 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf4114850 con 0x7f2cf410ab40 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cfba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf4114b50 con 0x7f2cf410a180 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cf9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cf9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36286/0 (socket says 172.21.15.27:36286) 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cf9000640 1 -- 172.21.15.27:0/1912625421 learned_addr learned my addr 172.21.15.27:0/1912625421 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.013+0000 7f2cf9a00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf410ab40 0x7f2cf41a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf3e00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 -- 172.21.15.27:0/1912625421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 msgr2=0x7f2cf419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 -- 172.21.15.27:0/1912625421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf410ab40 msgr2=0x7f2cf41a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf410ab40 0x7f2cf41a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 -- 172.21.15.27:0/1912625421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf41a6ca0 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf3e00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf9000640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e9c0 tx=0x7f2ce800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800cd80 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ce80044b0 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf41a6f90 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8010430 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.014+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf41a7450 con 0x7f2cf4108f90 2024-04-26T21:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.016+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf4110380 con 0x7f2cf4108f90 2024-04-26T21:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.018+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ce8003f00 con 0x7f2cf4108f90 2024-04-26T21:09:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.019+0000 7f2cf1600640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 0x7f2cdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.019+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2ce80d0970 con 0x7f2cf4108f90 2024-04-26T21:09:50.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.019+0000 7f2cf3e00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 0x7f2cdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.020+0000 7f2cf3e00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 0x7f2cdc0a6180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cf41a0590 tx=0x7f2ce0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.023+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ce8016330 con 0x7f2cf4108f90 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: pgmap v371: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 11 op/s 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] up:active 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-26T21:09:50.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:49 smithi094 ceph-mon[31273]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:09:50.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.262+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2cf4062d70 con 0x7f2cdc0a3cc0 2024-04-26T21:09:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.273+0000 7f2cf1600640 1 -- 172.21.15.27:0/1912625421 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2cf4062d70 con 0x7f2cdc0a3cc0 2024-04-26T21:09:50.275 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/f4bb00b1-1dac-4bc4-b772-30a03692e927 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 msgr2=0x7f2cdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 0x7f2cdc0a6180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cf41a0590 tx=0x7f2ce0005e50 comp rx=0 tx=0).stop 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 msgr2=0x7f2cf419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e9c0 tx=0x7f2ce800ee80 comp rx=0 tx=0).stop 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 shutdown_connections 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f2cdc0a3cc0 0x7f2cdc0a6180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2cf410a180 0x7f2cf419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf410ab40 0x7f2cf41a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.276+0000 7f2cfba00640 1 --2- 172.21.15.27:0/1912625421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cf4108f90 0x7f2cf419f080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.277+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 >> 172.21.15.27:0/1912625421 conn(0x7f2cf4077f30 msgr2=0x7f2cf410d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.277+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 shutdown_connections 2024-04-26T21:09:50.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.277+0000 7f2cfba00640 1 -- 172.21.15.27:0/1912625421 wait complete. 2024-04-26T21:09:50.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 -- 172.21.15.27:0/1080977925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 msgr2=0x7fa6181091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 --2- 172.21.15.27:0/1080977925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa6181091b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa608009ca0 tx=0x7fa60802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 -- 172.21.15.27:0/1080977925 shutdown_connections 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 --2- 172.21.15.27:0/1080977925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa61810a940 0x7fa6181111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 --2- 172.21.15.27:0/1080977925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa618109f80 0x7fa61810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 --2- 172.21.15.27:0/1080977925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa6181091b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.509+0000 7fa617e00640 1 -- 172.21.15.27:0/1080977925 >> 172.21.15.27:0/1080977925 conn(0x7fa618077d30 msgr2=0x7fa61807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.510+0000 7fa617e00640 1 -- 172.21.15.27:0/1080977925 shutdown_connections 2024-04-26T21:09:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.510+0000 7fa617e00640 1 -- 172.21.15.27:0/1080977925 wait complete. 2024-04-26T21:09:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 Processor -- start 2024-04-26T21:09:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 -- start start 2024-04-26T21:09:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 0x7fa61819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61810a940 0x7fa6181a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6181147c0 con 0x7fa618108d90 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa618114640 con 0x7fa61810a940 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.511+0000 7fa617e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa618114940 con 0x7fa618109f80 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36304/0 (socket says 172.21.15.27:36304) 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 -- 172.21.15.27:0/82737170 learned_addr learned my addr 172.21.15.27:0/82737170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616000640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 0x7fa61819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa617400640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61810a940 0x7fa6181a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 -- 172.21.15.27:0/82737170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 msgr2=0x7fa61819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 0x7fa61819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 -- 172.21.15.27:0/82737170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61810a940 msgr2=0x7fa6181a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61810a940 0x7fa6181a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 -- 172.21.15.27:0/82737170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6181a6af0 con 0x7fa618108d90 2024-04-26T21:09:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616000640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 0x7fa61819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.512+0000 7fa616a00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098d0 tx=0x7fa6080039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.513+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608004040 con 0x7fa618108d90 2024-04-26T21:09:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.513+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa608004670 con 0x7fa618108d90 2024-04-26T21:09:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.513+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6181a6d80 con 0x7fa618108d90 2024-04-26T21:09:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.513+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608030ce0 con 0x7fa618108d90 2024-04-26T21:09:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.513+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6181a7240 con 0x7fa618108d90 2024-04-26T21:09:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.514+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa618110180 con 0x7fa618108d90 2024-04-26T21:09:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.516+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa6080041e0 con 0x7fa618108d90 2024-04-26T21:09:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.517+0000 7fa61ca00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 0x7fa5fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.518+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa6080f19f0 con 0x7fa618108d90 2024-04-26T21:09:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.518+0000 7fa616000640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 0x7fa5fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.518+0000 7fa616000640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 0x7fa5fc0a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6181a03e0 tx=0x7fa60000b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.522+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6080be430 con 0x7fa618108d90 2024-04-26T21:09:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.780+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa61802dad0 con 0x7fa618108d90 2024-04-26T21:09:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.781+0000 7fa61ca00640 1 -- 172.21.15.27:0/82737170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7fa608004820 con 0x7fa618108d90 2024-04-26T21:09:50.783 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:50.783 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:09:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.784+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 msgr2=0x7fa5fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.784+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 0x7fa5fc0a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6181a03e0 tx=0x7fa60000b2e0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.784+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 msgr2=0x7fa61819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.784+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098d0 tx=0x7fa6080039f0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 shutdown_connections 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa5fc0a3cc0 0x7fa5fc0a6180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa618109f80 0x7fa61819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61810a940 0x7fa6181a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 --2- 172.21.15.27:0/82737170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa618108d90 0x7fa61819ef70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 >> 172.21.15.27:0/82737170 conn(0x7fa618077d30 msgr2=0x7fa61810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 shutdown_connections 2024-04-26T21:09:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:50.785+0000 7fa617e00640 1 -- 172.21.15.27:0/82737170 wait complete. 2024-04-26T21:09:50.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:50.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:50 smithi027 ceph-mon[28085]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:50.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:50 smithi027 ceph-mon[28085]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:50 smithi204 ceph-mon[32145]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:50 smithi204 ceph-mon[32145]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:50 smithi094 ceph-mon[31273]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:50 smithi094 ceph-mon[31273]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 -- 172.21.15.27:0/850470627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac106540 msgr2=0x7f0dac10cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 --2- 172.21.15.27:0/850470627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac106540 0x7f0dac10cdd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009ca0 tx=0x7f0da402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 -- 172.21.15.27:0/850470627 shutdown_connections 2024-04-26T21:09:51.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 --2- 172.21.15.27:0/850470627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac106540 0x7f0dac10cdd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 --2- 172.21.15.27:0/850470627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 --2- 172.21.15.27:0/850470627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac104990 0x7f0dac104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 -- 172.21.15.27:0/850470627 >> 172.21.15.27:0/850470627 conn(0x7f0dac0fff00 msgr2=0x7f0dac102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 -- 172.21.15.27:0/850470627 shutdown_connections 2024-04-26T21:09:51.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.026+0000 7f0dabe00640 1 -- 172.21.15.27:0/850470627 wait complete. 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.027+0000 7f0dabe00640 1 Processor -- start 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.027+0000 7f0dabe00640 1 -- start start 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac106540 0x7f0dac1a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1103c0 con 0x7f0dac104990 2024-04-26T21:09:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac110240 con 0x7f0dac105b80 2024-04-26T21:09:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0dabe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac110540 con 0x7f0dac106540 2024-04-26T21:09:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0daaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0daaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36320/0 (socket says 172.21.15.27:36320) 2024-04-26T21:09:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0daaa00640 1 -- 172.21.15.27:0/1284442174 learned_addr learned my addr 172.21.15.27:0/1284442174 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:51.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.028+0000 7f0daa000640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 -- 172.21.15.27:0/1284442174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac106540 msgr2=0x7f0dac1a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0dab400640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac106540 0x7f0dac1a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac106540 0x7f0dac1a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 -- 172.21.15.27:0/1284442174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 msgr2=0x7f0dac19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 -- 172.21.15.27:0/1284442174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac1a26a0 con 0x7f0dac104990 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daa000640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0daaa00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d950 tx=0x7f0d9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c017070 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9c009c30 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac1a2990 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00f620 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.029+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a2e90 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.031+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac10bd80 con 0x7f0dac104990 2024-04-26T21:09:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.033+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d9c01a070 con 0x7f0dac104990 2024-04-26T21:09:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.035+0000 7f0db0a00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 0x7f0d900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.035+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0d9c0cff00 con 0x7f0dac104990 2024-04-26T21:09:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.035+0000 7f0daa000640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 0x7f0d900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.036+0000 7f0daa000640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 0x7f0d900a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dac19bf90 tx=0x7f0d98005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.040+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0d9c009de0 con 0x7f0dac104990 2024-04-26T21:09:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.248+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dac02dd40 con 0x7f0dac104990 2024-04-26T21:09:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.250+0000 7f0db0a00640 1 -- 172.21.15.27:0/1284442174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f0d9c09c930 con 0x7f0dac104990 2024-04-26T21:09:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:51.252 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:09:42.028133+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 msgr2=0x7f0d900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 0x7f0d900a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dac19bf90 tx=0x7f0d98005e50 comp rx=0 tx=0).stop 2024-04-26T21:09:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 msgr2=0x7f0dac19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d950 tx=0x7f0d9c00de10 comp rx=0 tx=0).stop 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 shutdown_connections 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0d900a3c40 0x7f0d900a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac106540 0x7f0dac1a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dac105b80 0x7f0dac19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 --2- 172.21.15.27:0/1284442174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dac104990 0x7f0dac19aa80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.252+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 >> 172.21.15.27:0/1284442174 conn(0x7f0dac0fff00 msgr2=0x7f0dac108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.253+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 shutdown_connections 2024-04-26T21:09:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.253+0000 7f0dabe00640 1 -- 172.21.15.27:0/1284442174 wait complete. 2024-04-26T21:09:51.275 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:09:51.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 -- 172.21.15.27:0/3110383621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 msgr2=0x7f188c079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 --2- 172.21.15.27:0/3110383621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c079770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1884009ca0 tx=0x7f188402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 -- 172.21.15.27:0/3110383621 shutdown_connections 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 --2- 172.21.15.27:0/3110383621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c079770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 --2- 172.21.15.27:0/3110383621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 0x7f188c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 --2- 172.21.15.27:0/3110383621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 0x7f188c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.493+0000 7f188be00640 1 -- 172.21.15.27:0/3110383621 >> 172.21.15.27:0/3110383621 conn(0x7f188c0ffeb0 msgr2=0x7f188c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.494+0000 7f188be00640 1 -- 172.21.15.27:0/3110383621 shutdown_connections 2024-04-26T21:09:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.494+0000 7f188be00640 1 -- 172.21.15.27:0/3110383621 wait complete. 2024-04-26T21:09:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.494+0000 7f188be00640 1 Processor -- start 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 -- start start 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 0x7f188c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 0x7f188c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f188c114790 con 0x7f188c078ed0 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f188c114610 con 0x7f188c078530 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f188c114910 con 0x7f188c0772f0 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36340/0 (socket says 172.21.15.27:36340) 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.495+0000 7f188b400640 1 -- 172.21.15.27:0/1684057922 learned_addr learned my addr 172.21.15.27:0/1684057922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188aa00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 0x7f188c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f1883e00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 0x7f188c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 -- 172.21.15.27:0/1684057922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 msgr2=0x7f188c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 0x7f188c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 -- 172.21.15.27:0/1684057922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 msgr2=0x7f188c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 0x7f188c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 -- 172.21.15.27:0/1684057922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1a6a60 con 0x7f188c078ed0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188b400640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f188402aeb0 tx=0x7f1884003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884032ed0 con 0x7f188c078ed0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1884003e10 con 0x7f188c078ed0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c1a6cf0 con 0x7f188c078ed0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.496+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884030cd0 con 0x7f188c078ed0 2024-04-26T21:09:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.497+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1a71e0 con 0x7f188c078ed0 2024-04-26T21:09:51.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.498+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c1112d0 con 0x7f188c078ed0 2024-04-26T21:09:51.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.500+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1884004240 con 0x7f188c078ed0 2024-04-26T21:09:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.501+0000 7f1883400640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 0x7f18700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.501+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f18840f1b30 con 0x7f188c078ed0 2024-04-26T21:09:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.502+0000 7f188aa00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 0x7f18700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.502+0000 7f188aa00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 0x7f18700a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f187c005fd0 tx=0x7f187c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.505+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18840be570 con 0x7f188c078ed0 2024-04-26T21:09:51.609 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: 2024-04-26T21:09:51.354+0000 7f28fc400700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-26T21:09:51.609 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: 2024-04-26T21:09:51.354+0000 7f28fc400700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-26T21:09:51.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.713+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f188c062d70 con 0x7f18700a3c40 2024-04-26T21:09:51.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.740+0000 7f1883400640 1 -- 172.21.15.27:0/1684057922 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258808 (secure 0 0 0) 0x7f188c062d70 con 0x7f18700a3c40 2024-04-26T21:09:51.745 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 msgr2=0x7f18700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 0x7f18700a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f187c005fd0 tx=0x7f187c005850 comp rx=0 tx=0).stop 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 msgr2=0x7f188c1a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f188402aeb0 tx=0x7f1884003c20 comp rx=0 tx=0).stop 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 shutdown_connections 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f18700a3c40 0x7f18700a6100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f188c078ed0 0x7f188c1a63f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f188c078530 0x7f188c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 --2- 172.21.15.27:0/1684057922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f188c0772f0 0x7f188c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.746+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 >> 172.21.15.27:0/1684057922 conn(0x7f188c0ffeb0 msgr2=0x7f188c1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.747+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 shutdown_connections 2024-04-26T21:09:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:51.747+0000 7f188be00640 1 -- 172.21.15.27:0/1684057922 wait complete. 2024-04-26T21:09:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-26T21:09:50.045517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594986,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":137,"num_read_kb":120,"num_write":306,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395516,"kb_used_data":13068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124546308,"statfs":{"total":1151940427776,"available":1151535419392,"internally_reserved":0,"allocated":13381632,"data_stored":7234601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4618,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.015419"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037642+0000","last_change":"2024-04-26T21:09:41.072073+0000","last_active":"2024-04-26T21:09:42.037642+0000","last_peered":"2024-04-26T21:09:42.037642+0000","last_clean":"2024-04-26T21:09:42.037642+0000","last_became_active":"2024-04-26T21:09:41.071687+0000","last_became_peered":"2024-04-26T21:09:41.071687+0000","last_unstale":"2024-04-26T21:09:42.037642+0000","last_undegraded":"2024-04-26T21:09:42.037642+0000","last_fullsized":"2024-04-26T21:09:42.037642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:17.092134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104814+0000","last_change":"2024-04-26T21:09:41.063568+0000","last_active":"2024-04-26T21:09:42.104814+0000","last_peered":"2024-04-26T21:09:42.104814+0000","last_clean":"2024-04-26T21:09:42.104814+0000","last_became_active":"2024-04-26T21:09:41.061712+0000","last_became_peered":"2024-04-26T21:09:41.061712+0000","last_unstale":"2024-04-26T21:09:42.104814+0000","last_undegraded":"2024-04-26T21:09:42.104814+0000","last_fullsized":"2024-04-26T21:09:42.104814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:38.894168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103731+0000","last_change":"2024-04-26T21:09:41.068164+0000","last_active":"2024-04-26T21:09:42.103731+0000","last_peered":"2024-04-26T21:09:42.103731+0000","last_clean":"2024-04-26T21:09:42.103731+0000","last_became_active":"2024-04-26T21:09:41.067074+0000","last_became_peered":"2024-04-26T21:09:41.067074+0000","last_unstale":"2024-04-26T21:09:42.103731+0000","last_undegraded":"2024-04-26T21:09:42.103731+0000","last_fullsized":"2024-04-26T21:09:42.103731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.900193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101822+0000","last_change":"2024-04-26T21:09:41.076541+0000","last_active":"2024-04-26T21:09:42.101822+0000","last_peered":"2024-04-26T21:09:42.101822+0000","last_clean":"2024-04-26T21:09:42.101822+0000","last_became_active":"2024-04-26T21:09:41.074492+0000","last_became_peered":"2024-04-26T21:09:41.074492+0000","last_unstale":"2024-04-26T21:09:42.101822+0000","last_undegraded":"2024-04-26T21:09:42.101822+0000","last_fullsized":"2024-04-26T21:09:42.101822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:56.007862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103202+0000","last_change":"2024-04-26T21:09:41.070626+0000","last_active":"2024-04-26T21:09:42.103202+0000","last_peered":"2024-04-26T21:09:42.103202+0000","last_clean":"2024-04-26T21:09:42.103202+0000","last_became_active":"2024-04-26T21:09:41.070258+0000","last_became_peered":"2024-04-26T21:09:41.070258+0000","last_unstale":"2024-04-26T21:09:42.103202+0000","last_undegraded":"2024-04-26T21:09:42.103202+0000","last_fullsized":"2024-04-26T21:09:42.103202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:10.491491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104645+0000","last_change":"2024-04-26T21:09:41.066628+0000","last_active":"2024-04-26T21:09:42.104645+0000","last_peered":"2024-04-26T21:09:42.104645+0000","last_clean":"2024-04-26T21:09:42.104645+0000","last_became_active":"2024-04-26T21:09:41.066217+0000","last_became_peered":"2024-04-26T21:09:41.066217+0000","last_unstale":"2024-04-26T21:09:42.104645+0000","last_undegraded":"2024-04-26T21:09:42.104645+0000","last_fullsized":"2024-04-26T21:09:42.104645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:10:27.575166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037031+0000","last_change":"2024-04-26T21:09:41.069684+0000","last_active":"2024-04-26T21:09:42.037031+0000","last_peered":"2024-04-26T21:09:42.037031+0000","last_clean":"2024-04-26T21:09:42.037031+0000","last_became_active":"2024-04-26T21:09:41.069323+0000","last_became_peered":"2024-04-26T21:09:41.069323+0000","last_unstale":"2024-04-26T21:09:42.037031+0000","last_undegraded":"2024-04-26T21:09:42.037031+0000","last_fullsized":"2024-04-26T21:09:42.037031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.122606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100229+0000","last_change":"2024-04-26T21:09:41.069130+0000","last_active":"2024-04-26T21:09:42.100229+0000","last_peered":"2024-04-26T21:09:42.100229+0000","last_clean":"2024-04-26T21:09:42.100229+0000","last_became_active":"2024-04-26T21:09:41.068682+0000","last_became_peered":"2024-04-26T21:09:41.068682+0000","last_unstale":"2024-04-26T21:09:42.100229+0000","last_undegraded":"2024-04-26T21:09:42.100229+0000","last_fullsized":"2024-04-26T21:09:42.100229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:39.389562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102028+0000","last_change":"2024-04-26T21:09:41.073930+0000","last_active":"2024-04-26T21:09:42.102028+0000","last_peered":"2024-04-26T21:09:42.102028+0000","last_clean":"2024-04-26T21:09:42.102028+0000","last_became_active":"2024-04-26T21:09:41.072566+0000","last_became_peered":"2024-04-26T21:09:41.072566+0000","last_unstale":"2024-04-26T21:09:42.102028+0000","last_undegraded":"2024-04-26T21:09:42.102028+0000","last_fullsized":"2024-04-26T21:09:42.102028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:48.613310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105196+0000","last_change":"2024-04-26T21:09:41.065598+0000","last_active":"2024-04-26T21:09:42.105196+0000","last_peered":"2024-04-26T21:09:42.105196+0000","last_clean":"2024-04-26T21:09:42.105196+0000","last_became_active":"2024-04-26T21:09:41.064977+0000","last_became_peered":"2024-04-26T21:09:41.064977+0000","last_unstale":"2024-04-26T21:09:42.105196+0000","last_undegraded":"2024-04-26T21:09:42.105196+0000","last_fullsized":"2024-04-26T21:09:42.105196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:39.776967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038255+0000","last_change":"2024-04-26T21:09:41.077939+0000","last_active":"2024-04-26T21:09:42.038255+0000","last_peered":"2024-04-26T21:09:42.038255+0000","last_clean":"2024-04-26T21:09:42.038255+0000","last_became_active":"2024-04-26T21:09:41.077641+0000","last_became_peered":"2024-04-26T21:09:41.077641+0000","last_unstale":"2024-04-26T21:09:42.038255+0000","last_undegraded":"2024-04-26T21:09:42.038255+0000","last_fullsized":"2024-04-26T21:09:42.038255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:47.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101331+0000","last_change":"2024-04-26T21:09:41.075109+0000","last_active":"2024-04-26T21:09:42.101331+0000","last_peered":"2024-04-26T21:09:42.101331+0000","last_clean":"2024-04-26T21:09:42.101331+0000","last_became_active":"2024-04-26T21:09:41.073866+0000","last_became_peered":"2024-04-26T21:09:41.073866+0000","last_unstale":"2024-04-26T21:09:42.101331+0000","last_undegraded":"2024-04-26T21:09:42.101331+0000","last_fullsized":"2024-04-26T21:09:42.101331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:16.786155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105287+0000","last_change":"2024-04-26T21:09:41.074934+0000","last_active":"2024-04-26T21:09:42.105287+0000","last_peered":"2024-04-26T21:09:42.105287+0000","last_clean":"2024-04-26T21:09:42.105287+0000","last_became_active":"2024-04-26T21:09:41.072388+0000","last_became_peered":"2024-04-26T21:09:41.072388+0000","last_unstale":"2024-04-26T21:09:42.105287+0000","last_undegraded":"2024-04-26T21:09:42.105287+0000","last_fullsized":"2024-04-26T21:09:42.105287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:23.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100481+0000","last_change":"2024-04-26T21:09:41.062598+0000","last_active":"2024-04-26T21:09:42.100481+0000","last_peered":"2024-04-26T21:09:42.100481+0000","last_clean":"2024-04-26T21:09:42.100481+0000","last_became_active":"2024-04-26T21:09:41.061828+0000","last_became_peered":"2024-04-26T21:09:41.061828+0000","last_unstale":"2024-04-26T21:09:42.100481+0000","last_undegraded":"2024-04-26T21:09:42.100481+0000","last_fullsized":"2024-04-26T21:09:42.100481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:52.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100659+0000","last_change":"2024-04-26T21:09:41.064874+0000","last_active":"2024-04-26T21:09:42.100659+0000","last_peered":"2024-04-26T21:09:42.100659+0000","last_clean":"2024-04-26T21:09:42.100659+0000","last_became_active":"2024-04-26T21:09:41.060190+0000","last_became_peered":"2024-04-26T21:09:41.060190+0000","last_unstale":"2024-04-26T21:09:42.100659+0000","last_undegraded":"2024-04-26T21:09:42.100659+0000","last_fullsized":"2024-04-26T21:09:42.100659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101169+0000","last_change":"2024-04-26T21:09:41.067343+0000","last_active":"2024-04-26T21:09:42.101169+0000","last_peered":"2024-04-26T21:09:42.101169+0000","last_clean":"2024-04-26T21:09:42.101169+0000","last_became_active":"2024-04-26T21:09:41.066248+0000","last_became_peered":"2024-04-26T21:09:41.066248+0000","last_unstale":"2024-04-26T21:09:42.101169+0000","last_undegraded":"2024-04-26T21:09:42.101169+0000","last_fullsized":"2024-04-26T21:09:42.101169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:40.476089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038084+0000","last_change":"2024-04-26T21:09:41.079414+0000","last_active":"2024-04-26T21:09:42.038084+0000","last_peered":"2024-04-26T21:09:42.038084+0000","last_clean":"2024-04-26T21:09:42.038084+0000","last_became_active":"2024-04-26T21:09:41.078949+0000","last_became_peered":"2024-04-26T21:09:41.078949+0000","last_unstale":"2024-04-26T21:09:42.038084+0000","last_undegraded":"2024-04-26T21:09:42.038084+0000","last_fullsized":"2024-04-26T21:09:42.038084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:19.947290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036239+0000","last_change":"2024-04-26T21:09:41.069376+0000","last_active":"2024-04-26T21:09:42.036239+0000","last_peered":"2024-04-26T21:09:42.036239+0000","last_clean":"2024-04-26T21:09:42.036239+0000","last_became_active":"2024-04-26T21:09:41.069049+0000","last_became_peered":"2024-04-26T21:09:41.069049+0000","last_unstale":"2024-04-26T21:09:42.036239+0000","last_undegraded":"2024-04-26T21:09:42.036239+0000","last_fullsized":"2024-04-26T21:09:42.036239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:25.058292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104058+0000","last_change":"2024-04-26T21:09:41.072485+0000","last_active":"2024-04-26T21:09:42.104058+0000","last_peered":"2024-04-26T21:09:42.104058+0000","last_clean":"2024-04-26T21:09:42.104058+0000","last_became_active":"2024-04-26T21:09:41.070990+0000","last_became_peered":"2024-04-26T21:09:41.070990+0000","last_unstale":"2024-04-26T21:09:42.104058+0000","last_undegraded":"2024-04-26T21:09:42.104058+0000","last_fullsized":"2024-04-26T21:09:42.104058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:40.648097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.501095+0000","last_change":"2024-04-26T21:09:40.391208+0000","last_active":"2024-04-26T21:09:48.501095+0000","last_peered":"2024-04-26T21:09:48.501095+0000","last_clean":"2024-04-26T21:09:48.501095+0000","last_became_active":"2024-04-26T21:09:40.390216+0000","last_became_peered":"2024-04-26T21:09:40.390216+0000","last_unstale":"2024-04-26T21:09:48.501095+0000","last_undegraded":"2024-04-26T21:09:48.501095+0000","last_fullsized":"2024-04-26T21:09:48.501095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:40.516383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100509+0000","last_change":"2024-04-26T21:09:41.084249+0000","last_active":"2024-04-26T21:09:42.100509+0000","last_peered":"2024-04-26T21:09:42.100509+0000","last_clean":"2024-04-26T21:09:42.100509+0000","last_became_active":"2024-04-26T21:09:41.083580+0000","last_became_peered":"2024-04-26T21:09:41.083580+0000","last_unstale":"2024-04-26T21:09:42.100509+0000","last_undegraded":"2024-04-26T21:09:42.100509+0000","last_fullsized":"2024-04-26T21:09:42.100509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:02.615403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:43.526770+0000","last_change":"2024-04-26T21:09:40.389376+0000","last_active":"2024-04-26T21:09:43.526770+0000","last_peered":"2024-04-26T21:09:43.526770+0000","last_clean":"2024-04-26T21:09:43.526770+0000","last_became_active":"2024-04-26T21:09:40.388761+0000","last_became_peered":"2024-04-26T21:09:40.388761+0000","last_unstale":"2024-04-26T21:09:43.526770+0000","last_undegraded":"2024-04-26T21:09:43.526770+0000","last_fullsized":"2024-04-26T21:09:43.526770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:39.940524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102397+0000","last_change":"2024-04-26T21:09:41.073193+0000","last_active":"2024-04-26T21:09:42.102397+0000","last_peered":"2024-04-26T21:09:42.102397+0000","last_clean":"2024-04-26T21:09:42.102397+0000","last_became_active":"2024-04-26T21:09:41.071569+0000","last_became_peered":"2024-04-26T21:09:41.071569+0000","last_unstale":"2024-04-26T21:09:42.102397+0000","last_undegraded":"2024-04-26T21:09:42.102397+0000","last_fullsized":"2024-04-26T21:09:42.102397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:40.078386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102742+0000","last_change":"2024-04-26T21:09:40.394188+0000","last_active":"2024-04-26T21:09:42.102742+0000","last_peered":"2024-04-26T21:09:42.102742+0000","last_clean":"2024-04-26T21:09:42.102742+0000","last_became_active":"2024-04-26T21:09:40.393854+0000","last_became_peered":"2024-04-26T21:09:40.393854+0000","last_unstale":"2024-04-26T21:09:42.102742+0000","last_undegraded":"2024-04-26T21:09:42.102742+0000","last_fullsized":"2024-04-26T21:09:42.102742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:30.232353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103098+0000","last_change":"2024-04-26T21:09:41.068545+0000","last_active":"2024-04-26T21:09:42.103098+0000","last_peered":"2024-04-26T21:09:42.103098+0000","last_clean":"2024-04-26T21:09:42.103098+0000","last_became_active":"2024-04-26T21:09:41.066805+0000","last_became_peered":"2024-04-26T21:09:41.066805+0000","last_unstale":"2024-04-26T21:09:42.103098+0000","last_undegraded":"2024-04-26T21:09:42.103098+0000","last_fullsized":"2024-04-26T21:09:42.103098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:32.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105271+0000","last_change":"2024-04-26T21:09:40.382727+0000","last_active":"2024-04-26T21:09:42.105271+0000","last_peered":"2024-04-26T21:09:42.105271+0000","last_clean":"2024-04-26T21:09:42.105271+0000","last_became_active":"2024-04-26T21:09:40.380205+0000","last_became_peered":"2024-04-26T21:09:40.380205+0000","last_unstale":"2024-04-26T21:09:42.105271+0000","last_undegraded":"2024-04-26T21:09:42.105271+0000","last_fullsized":"2024-04-26T21:09:42.105271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:32.981478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105299+0000","last_change":"2024-04-26T21:09:41.075664+0000","last_active":"2024-04-26T21:09:42.105299+0000","last_peered":"2024-04-26T21:09:42.105299+0000","last_clean":"2024-04-26T21:09:42.105299+0000","last_became_active":"2024-04-26T21:09:41.075213+0000","last_became_peered":"2024-04-26T21:09:41.075213+0000","last_unstale":"2024-04-26T21:09:42.105299+0000","last_undegraded":"2024-04-26T21:09:42.105299+0000","last_fullsized":"2024-04-26T21:09:42.105299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:11.832226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105932+0000","last_change":"2024-04-26T21:09:40.395282+0000","last_active":"2024-04-26T21:09:42.105932+0000","last_peered":"2024-04-26T21:09:42.105932+0000","last_clean":"2024-04-26T21:09:42.105932+0000","last_became_active":"2024-04-26T21:09:40.394858+0000","last_became_peered":"2024-04-26T21:09:40.394858+0000","last_unstale":"2024-04-26T21:09:42.105932+0000","last_undegraded":"2024-04-26T21:09:42.105932+0000","last_fullsized":"2024-04-26T21:09:42.105932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:38.330771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102517+0000","last_change":"2024-04-26T21:09:41.064853+0000","last_active":"2024-04-26T21:09:42.102517+0000","last_peered":"2024-04-26T21:09:42.102517+0000","last_clean":"2024-04-26T21:09:42.102517+0000","last_became_active":"2024-04-26T21:09:41.064236+0000","last_became_peered":"2024-04-26T21:09:41.064236+0000","last_unstale":"2024-04-26T21:09:42.102517+0000","last_undegraded":"2024-04-26T21:09:42.102517+0000","last_fullsized":"2024-04-26T21:09:42.102517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:05.549027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105537+0000","last_change":"2024-04-26T21:09:40.390857+0000","last_active":"2024-04-26T21:09:42.105537+0000","last_peered":"2024-04-26T21:09:42.105537+0000","last_clean":"2024-04-26T21:09:42.105537+0000","last_became_active":"2024-04-26T21:09:40.390513+0000","last_became_peered":"2024-04-26T21:09:40.390513+0000","last_unstale":"2024-04-26T21:09:42.105537+0000","last_undegraded":"2024-04-26T21:09:42.105537+0000","last_fullsized":"2024-04-26T21:09:42.105537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.106177+0000","last_change":"2024-04-26T21:09:41.074105+0000","last_active":"2024-04-26T21:09:42.106177+0000","last_peered":"2024-04-26T21:09:42.106177+0000","last_clean":"2024-04-26T21:09:42.106177+0000","last_became_active":"2024-04-26T21:09:41.073645+0000","last_became_peered":"2024-04-26T21:09:41.073645+0000","last_unstale":"2024-04-26T21:09:42.106177+0000","last_undegraded":"2024-04-26T21:09:42.106177+0000","last_fullsized":"2024-04-26T21:09:42.106177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:04.118506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.585807+0000","last_change":"2024-04-26T21:09:40.390893+0000","last_active":"2024-04-26T21:09:46.585807+0000","last_peered":"2024-04-26T21:09:46.585807+0000","last_clean":"2024-04-26T21:09:46.585807+0000","last_became_active":"2024-04-26T21:09:40.390519+0000","last_became_peered":"2024-04-26T21:09:40.390519+0000","last_unstale":"2024-04-26T21:09:46.585807+0000","last_undegraded":"2024-04-26T21:09:46.585807+0000","last_fullsized":"2024-04-26T21:09:46.585807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:40.963614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101057+0000","last_change":"2024-04-26T21:09:41.074751+0000","last_active":"2024-04-26T21:09:42.101057+0000","last_peered":"2024-04-26T21:09:42.101057+0000","last_clean":"2024-04-26T21:09:42.101057+0000","last_became_active":"2024-04-26T21:09:41.074330+0000","last_became_peered":"2024-04-26T21:09:41.074330+0000","last_unstale":"2024-04-26T21:09:42.101057+0000","last_undegraded":"2024-04-26T21:09:42.101057+0000","last_fullsized":"2024-04-26T21:09:42.101057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:47.759543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102206+0000","last_change":"2024-04-26T21:09:40.389971+0000","last_active":"2024-04-26T21:09:42.102206+0000","last_peered":"2024-04-26T21:09:42.102206+0000","last_clean":"2024-04-26T21:09:42.102206+0000","last_became_active":"2024-04-26T21:09:40.389093+0000","last_became_peered":"2024-04-26T21:09:40.389093+0000","last_unstale":"2024-04-26T21:09:42.102206+0000","last_undegraded":"2024-04-26T21:09:42.102206+0000","last_fullsized":"2024-04-26T21:09:42.102206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:18.838994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.035164+0000","last_change":"2024-04-26T21:09:41.068903+0000","last_active":"2024-04-26T21:09:42.035164+0000","last_peered":"2024-04-26T21:09:42.035164+0000","last_clean":"2024-04-26T21:09:42.035164+0000","last_became_active":"2024-04-26T21:09:41.068585+0000","last_became_peered":"2024-04-26T21:09:41.068585+0000","last_unstale":"2024-04-26T21:09:42.035164+0000","last_undegraded":"2024-04-26T21:09:42.035164+0000","last_fullsized":"2024-04-26T21:09:42.035164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:18.372213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.575723+0000","last_change":"2024-04-26T21:09:40.391501+0000","last_active":"2024-04-26T21:09:45.575723+0000","last_peered":"2024-04-26T21:09:45.575723+0000","last_clean":"2024-04-26T21:09:45.575723+0000","last_became_active":"2024-04-26T21:09:40.390434+0000","last_became_peered":"2024-04-26T21:09:40.390434+0000","last_unstale":"2024-04-26T21:09:45.575723+0000","last_undegraded":"2024-04-26T21:09:45.575723+0000","last_fullsized":"2024-04-26T21:09:45.575723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:11.854716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102675+0000","last_change":"2024-04-26T21:09:41.064937+0000","last_active":"2024-04-26T21:09:42.102675+0000","last_peered":"2024-04-26T21:09:42.102675+0000","last_clean":"2024-04-26T21:09:42.102675+0000","last_became_active":"2024-04-26T21:09:41.063667+0000","last_became_peered":"2024-04-26T21:09:41.063667+0000","last_unstale":"2024-04-26T21:09:42.102675+0000","last_undegraded":"2024-04-26T21:09:42.102675+0000","last_fullsized":"2024-04-26T21:09:42.102675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.380869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.573885+0000","last_change":"2024-04-26T21:09:40.394499+0000","last_active":"2024-04-26T21:09:45.573885+0000","last_peered":"2024-04-26T21:09:45.573885+0000","last_clean":"2024-04-26T21:09:45.573885+0000","last_became_active":"2024-04-26T21:09:40.394052+0000","last_became_peered":"2024-04-26T21:09:40.394052+0000","last_unstale":"2024-04-26T21:09:45.573885+0000","last_undegraded":"2024-04-26T21:09:45.573885+0000","last_fullsized":"2024-04-26T21:09:45.573885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:51.540913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102187+0000","last_change":"2024-04-26T21:09:41.064559+0000","last_active":"2024-04-26T21:09:42.102187+0000","last_peered":"2024-04-26T21:09:42.102187+0000","last_clean":"2024-04-26T21:09:42.102187+0000","last_became_active":"2024-04-26T21:09:41.063117+0000","last_became_peered":"2024-04-26T21:09:41.063117+0000","last_unstale":"2024-04-26T21:09:42.102187+0000","last_undegraded":"2024-04-26T21:09:42.102187+0000","last_fullsized":"2024-04-26T21:09:42.102187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:34.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105364+0000","last_change":"2024-04-26T21:09:40.390507+0000","last_active":"2024-04-26T21:09:42.105364+0000","last_peered":"2024-04-26T21:09:42.105364+0000","last_clean":"2024-04-26T21:09:42.105364+0000","last_became_active":"2024-04-26T21:09:40.388503+0000","last_became_peered":"2024-04-26T21:09:40.388503+0000","last_unstale":"2024-04-26T21:09:42.105364+0000","last_undegraded":"2024-04-26T21:09:42.105364+0000","last_fullsized":"2024-04-26T21:09:42.105364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:40.432667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102932+0000","last_change":"2024-04-26T21:09:41.065188+0000","last_active":"2024-04-26T21:09:42.102932+0000","last_peered":"2024-04-26T21:09:42.102932+0000","last_clean":"2024-04-26T21:09:42.102932+0000","last_became_active":"2024-04-26T21:09:41.061218+0000","last_became_peered":"2024-04-26T21:09:41.061218+0000","last_unstale":"2024-04-26T21:09:42.102932+0000","last_undegraded":"2024-04-26T21:09:42.102932+0000","last_fullsized":"2024-04-26T21:09:42.102932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:42.087561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.573014+0000","last_change":"2024-04-26T21:09:40.393372+0000","last_active":"2024-04-26T21:09:45.573014+0000","last_peered":"2024-04-26T21:09:45.573014+0000","last_clean":"2024-04-26T21:09:45.573014+0000","last_became_active":"2024-04-26T21:09:40.392631+0000","last_became_peered":"2024-04-26T21:09:40.392631+0000","last_unstale":"2024-04-26T21:09:45.573014+0000","last_undegraded":"2024-04-26T21:09:45.573014+0000","last_fullsized":"2024-04-26T21:09:45.573014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.347781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102442+0000","last_change":"2024-04-26T21:09:41.063711+0000","last_active":"2024-04-26T21:09:42.102442+0000","last_peered":"2024-04-26T21:09:42.102442+0000","last_clean":"2024-04-26T21:09:42.102442+0000","last_became_active":"2024-04-26T21:09:41.063330+0000","last_became_peered":"2024-04-26T21:09:41.063330+0000","last_unstale":"2024-04-26T21:09:42.102442+0000","last_undegraded":"2024-04-26T21:09:42.102442+0000","last_fullsized":"2024-04-26T21:09:42.102442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:28.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.589498+0000","last_change":"2024-04-26T21:09:40.404407+0000","last_active":"2024-04-26T21:09:46.589498+0000","last_peered":"2024-04-26T21:09:46.589498+0000","last_clean":"2024-04-26T21:09:46.589498+0000","last_became_active":"2024-04-26T21:09:40.402751+0000","last_became_peered":"2024-04-26T21:09:40.402751+0000","last_unstale":"2024-04-26T21:09:46.589498+0000","last_undegraded":"2024-04-26T21:09:46.589498+0000","last_fullsized":"2024-04-26T21:09:46.589498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:32.171086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104657+0000","last_change":"2024-04-26T21:09:41.079310+0000","last_active":"2024-04-26T21:09:42.104657+0000","last_peered":"2024-04-26T21:09:42.104657+0000","last_clean":"2024-04-26T21:09:42.104657+0000","last_became_active":"2024-04-26T21:09:41.077848+0000","last_became_peered":"2024-04-26T21:09:41.077848+0000","last_unstale":"2024-04-26T21:09:42.104657+0000","last_undegraded":"2024-04-26T21:09:42.104657+0000","last_fullsized":"2024-04-26T21:09:42.104657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:51.745349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105842+0000","last_change":"2024-04-26T21:09:40.391216+0000","last_active":"2024-04-26T21:09:42.105842+0000","last_peered":"2024-04-26T21:09:42.105842+0000","last_clean":"2024-04-26T21:09:42.105842+0000","last_became_active":"2024-04-26T21:09:40.390753+0000","last_became_peered":"2024-04-26T21:09:40.390753+0000","last_unstale":"2024-04-26T21:09:42.105842+0000","last_undegraded":"2024-04-26T21:09:42.105842+0000","last_fullsized":"2024-04-26T21:09:42.105842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:24.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037992+0000","last_change":"2024-04-26T21:09:41.079881+0000","last_active":"2024-04-26T21:09:42.037992+0000","last_peered":"2024-04-26T21:09:42.037992+0000","last_clean":"2024-04-26T21:09:42.037992+0000","last_became_active":"2024-04-26T21:09:41.079506+0000","last_became_peered":"2024-04-26T21:09:41.079506+0000","last_unstale":"2024-04-26T21:09:42.037992+0000","last_undegraded":"2024-04-26T21:09:42.037992+0000","last_fullsized":"2024-04-26T21:09:42.037992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:56.300294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.586525+0000","last_change":"2024-04-26T21:09:40.391458+0000","last_active":"2024-04-26T21:09:46.586525+0000","last_peered":"2024-04-26T21:09:46.586525+0000","last_clean":"2024-04-26T21:09:46.586525+0000","last_became_active":"2024-04-26T21:09:40.388494+0000","last_became_peered":"2024-04-26T21:09:40.388494+0000","last_unstale":"2024-04-26T21:09:46.586525+0000","last_undegraded":"2024-04-26T21:09:46.586525+0000","last_fullsized":"2024-04-26T21:09:46.586525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:07.523885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036841+0000","last_change":"2024-04-26T21:09:41.068278+0000","last_active":"2024-04-26T21:09:42.036841+0000","last_peered":"2024-04-26T21:09:42.036841+0000","last_clean":"2024-04-26T21:09:42.036841+0000","last_became_active":"2024-04-26T21:09:41.067865+0000","last_became_peered":"2024-04-26T21:09:41.067865+0000","last_unstale":"2024-04-26T21:09:42.036841+0000","last_undegraded":"2024-04-26T21:09:42.036841+0000","last_fullsized":"2024-04-26T21:09:42.036841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:19.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'10","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.574382+0000","last_change":"2024-04-26T21:09:40.394390+0000","last_active":"2024-04-26T21:09:45.574382+0000","last_peered":"2024-04-26T21:09:45.574382+0000","last_clean":"2024-04-26T21:09:45.574382+0000","last_became_active":"2024-04-26T21:09:40.389728+0000","last_became_peered":"2024-04-26T21:09:40.389728+0000","last_unstale":"2024-04-26T21:09:45.574382+0000","last_undegraded":"2024-04-26T21:09:45.574382+0000","last_fullsized":"2024-04-26T21:09:45.574382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:42.301974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105384+0000","last_change":"2024-04-26T21:09:41.075703+0000","last_active":"2024-04-26T21:09:42.105384+0000","last_peered":"2024-04-26T21:09:42.105384+0000","last_clean":"2024-04-26T21:09:42.105384+0000","last_became_active":"2024-04-26T21:09:41.075310+0000","last_became_peered":"2024-04-26T21:09:41.075310+0000","last_unstale":"2024-04-26T21:09:42.105384+0000","last_undegraded":"2024-04-26T21:09:42.105384+0000","last_fullsized":"2024-04-26T21:09:42.105384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:20.030941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104136+0000","last_change":"2024-04-26T21:09:40.381293+0000","last_active":"2024-04-26T21:09:42.104136+0000","last_peered":"2024-04-26T21:09:42.104136+0000","last_clean":"2024-04-26T21:09:42.104136+0000","last_became_active":"2024-04-26T21:09:40.380287+0000","last_became_peered":"2024-04-26T21:09:40.380287+0000","last_unstale":"2024-04-26T21:09:42.104136+0000","last_undegraded":"2024-04-26T21:09:42.104136+0000","last_fullsized":"2024-04-26T21:09:42.104136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:47.479546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037097+0000","last_change":"2024-04-26T21:09:41.077928+0000","last_active":"2024-04-26T21:09:42.037097+0000","last_peered":"2024-04-26T21:09:42.037097+0000","last_clean":"2024-04-26T21:09:42.037097+0000","last_became_active":"2024-04-26T21:09:41.076611+0000","last_became_peered":"2024-04-26T21:09:41.076611+0000","last_unstale":"2024-04-26T21:09:42.037097+0000","last_undegraded":"2024-04-26T21:09:42.037097+0000","last_fullsized":"2024-04-26T21:09:42.037097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:43.893631+0000","last_change":"2024-04-26T21:09:40.394161+0000","last_active":"2024-04-26T21:09:43.893631+0000","last_peered":"2024-04-26T21:09:43.893631+0000","last_clean":"2024-04-26T21:09:43.893631+0000","last_became_active":"2024-04-26T21:09:40.393277+0000","last_became_peered":"2024-04-26T21:09:40.393277+0000","last_unstale":"2024-04-26T21:09:43.893631+0000","last_undegraded":"2024-04-26T21:09:43.893631+0000","last_fullsized":"2024-04-26T21:09:43.893631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:32.408080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104336+0000","last_change":"2024-04-26T21:09:41.070926+0000","last_active":"2024-04-26T21:09:42.104336+0000","last_peered":"2024-04-26T21:09:42.104336+0000","last_clean":"2024-04-26T21:09:42.104336+0000","last_became_active":"2024-04-26T21:09:41.070260+0000","last_became_peered":"2024-04-26T21:09:41.070260+0000","last_unstale":"2024-04-26T21:09:42.104336+0000","last_undegraded":"2024-04-26T21:09:42.104336+0000","last_fullsized":"2024-04-26T21:09:42.104336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:22.916960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037336+0000","last_change":"2024-04-26T21:09:40.395088+0000","last_active":"2024-04-26T21:09:42.037336+0000","last_peered":"2024-04-26T21:09:42.037336+0000","last_clean":"2024-04-26T21:09:42.037336+0000","last_became_active":"2024-04-26T21:09:40.393921+0000","last_became_peered":"2024-04-26T21:09:40.393921+0000","last_unstale":"2024-04-26T21:09:42.037336+0000","last_undegraded":"2024-04-26T21:09:42.037336+0000","last_fullsized":"2024-04-26T21:09:42.037336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:30.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105610+0000","last_change":"2024-04-26T21:09:41.064834+0000","last_active":"2024-04-26T21:09:42.105610+0000","last_peered":"2024-04-26T21:09:42.105610+0000","last_clean":"2024-04-26T21:09:42.105610+0000","last_became_active":"2024-04-26T21:09:41.063875+0000","last_became_peered":"2024-04-26T21:09:41.063875+0000","last_unstale":"2024-04-26T21:09:42.105610+0000","last_undegraded":"2024-04-26T21:09:42.105610+0000","last_fullsized":"2024-04-26T21:09:42.105610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:48.438740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103236+0000","last_change":"2024-04-26T21:09:41.075181+0000","last_active":"2024-04-26T21:09:42.103236+0000","last_peered":"2024-04-26T21:09:42.103236+0000","last_clean":"2024-04-26T21:09:42.103236+0000","last_became_active":"2024-04-26T21:09:41.073137+0000","last_became_peered":"2024-04-26T21:09:41.073137+0000","last_unstale":"2024-04-26T21:09:42.103236+0000","last_undegraded":"2024-04-26T21:09:42.103236+0000","last_fullsized":"2024-04-26T21:09:42.103236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:08.609048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.589629+0000","last_change":"2024-04-26T21:09:40.404159+0000","last_active":"2024-04-26T21:09:46.589629+0000","last_peered":"2024-04-26T21:09:46.589629+0000","last_clean":"2024-04-26T21:09:46.589629+0000","last_became_active":"2024-04-26T21:09:40.403828+0000","last_became_peered":"2024-04-26T21:09:40.403828+0000","last_unstale":"2024-04-26T21:09:46.589629+0000","last_undegraded":"2024-04-26T21:09:46.589629+0000","last_fullsized":"2024-04-26T21:09:46.589629+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:51.245405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103873+0000","last_change":"2024-04-26T21:09:41.068913+0000","last_active":"2024-04-26T21:09:42.103873+0000","last_peered":"2024-04-26T21:09:42.103873+0000","last_clean":"2024-04-26T21:09:42.103873+0000","last_became_active":"2024-04-26T21:09:41.068534+0000","last_became_peered":"2024-04-26T21:09:41.068534+0000","last_unstale":"2024-04-26T21:09:42.103873+0000","last_undegraded":"2024-04-26T21:09:42.103873+0000","last_fullsized":"2024-04-26T21:09:42.103873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:39.739853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.571169+0000","last_change":"2024-04-26T21:09:40.387055+0000","last_active":"2024-04-26T21:09:45.571169+0000","last_peered":"2024-04-26T21:09:45.571169+0000","last_clean":"2024-04-26T21:09:45.571169+0000","last_became_active":"2024-04-26T21:09:40.386684+0000","last_became_peered":"2024-04-26T21:09:40.386684+0000","last_unstale":"2024-04-26T21:09:45.571169+0000","last_undegraded":"2024-04-26T21:09:45.571169+0000","last_fullsized":"2024-04-26T21:09:45.571169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:02.871494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036617+0000","last_change":"2024-04-26T21:08:05.429394+0000","last_active":"2024-04-26T21:09:42.036617+0000","last_peered":"2024-04-26T21:09:42.036617+0000","last_clean":"2024-04-26T21:09:42.036617+0000","last_became_active":"2024-04-26T21:08:05.428055+0000","last_became_peered":"2024-04-26T21:08:05.428055+0000","last_unstale":"2024-04-26T21:09:42.036617+0000","last_undegraded":"2024-04-26T21:09:42.036617+0000","last_fullsized":"2024-04-26T21:09:42.036617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_clean_scrub_stamp":"2024-04-26T21:03:22.595431+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:23.104153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104544+0000","last_change":"2024-04-26T21:09:41.080213+0000","last_active":"2024-04-26T21:09:42.104544+0000","last_peered":"2024-04-26T21:09:42.104544+0000","last_clean":"2024-04-26T21:09:42.104544+0000","last_became_active":"2024-04-26T21:09:41.076313+0000","last_became_peered":"2024-04-26T21:09:41.076313+0000","last_unstale":"2024-04-26T21:09:42.104544+0000","last_undegraded":"2024-04-26T21:09:42.104544+0000","last_fullsized":"2024-04-26T21:09:42.104544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:48.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036872+0000","last_change":"2024-04-26T21:09:40.387563+0000","last_active":"2024-04-26T21:09:42.036872+0000","last_peered":"2024-04-26T21:09:42.036872+0000","last_clean":"2024-04-26T21:09:42.036872+0000","last_became_active":"2024-04-26T21:09:40.383243+0000","last_became_peered":"2024-04-26T21:09:40.383243+0000","last_unstale":"2024-04-26T21:09:42.036872+0000","last_undegraded":"2024-04-26T21:09:42.036872+0000","last_fullsized":"2024-04-26T21:09:42.036872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:16.905074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102599+0000","last_change":"2024-04-26T21:09:40.398078+0000","last_active":"2024-04-26T21:09:42.102599+0000","last_peered":"2024-04-26T21:09:42.102599+0000","last_clean":"2024-04-26T21:09:42.102599+0000","last_became_active":"2024-04-26T21:09:40.397007+0000","last_became_peered":"2024-04-26T21:09:40.397007+0000","last_unstale":"2024-04-26T21:09:42.102599+0000","last_undegraded":"2024-04-26T21:09:42.102599+0000","last_fullsized":"2024-04-26T21:09:42.102599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:18.374678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105027+0000","last_change":"2024-04-26T21:09:41.063430+0000","last_active":"2024-04-26T21:09:42.105027+0000","last_peered":"2024-04-26T21:09:42.105027+0000","last_clean":"2024-04-26T21:09:42.105027+0000","last_became_active":"2024-04-26T21:09:41.062591+0000","last_became_peered":"2024-04-26T21:09:41.062591+0000","last_unstale":"2024-04-26T21:09:42.105027+0000","last_undegraded":"2024-04-26T21:09:42.105027+0000","last_fullsized":"2024-04-26T21:09:42.105027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:58.874576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.607299+0000","last_change":"2024-04-26T21:09:40.378465+0000","last_active":"2024-04-26T21:09:47.607299+0000","last_peered":"2024-04-26T21:09:47.607299+0000","last_clean":"2024-04-26T21:09:47.607299+0000","last_became_active":"2024-04-26T21:09:40.378089+0000","last_became_peered":"2024-04-26T21:09:40.378089+0000","last_unstale":"2024-04-26T21:09:47.607299+0000","last_undegraded":"2024-04-26T21:09:47.607299+0000","last_fullsized":"2024-04-26T21:09:47.607299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:56.425077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101610+0000","last_change":"2024-04-26T21:09:41.080446+0000","last_active":"2024-04-26T21:09:42.101610+0000","last_peered":"2024-04-26T21:09:42.101610+0000","last_clean":"2024-04-26T21:09:42.101610+0000","last_became_active":"2024-04-26T21:09:41.074363+0000","last_became_peered":"2024-04-26T21:09:41.074363+0000","last_unstale":"2024-04-26T21:09:42.101610+0000","last_undegraded":"2024-04-26T21:09:42.101610+0000","last_fullsized":"2024-04-26T21:09:42.101610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:14.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.568444+0000","last_change":"2024-04-26T21:09:40.403106+0000","last_active":"2024-04-26T21:09:45.568444+0000","last_peered":"2024-04-26T21:09:45.568444+0000","last_clean":"2024-04-26T21:09:45.568444+0000","last_became_active":"2024-04-26T21:09:40.402630+0000","last_became_peered":"2024-04-26T21:09:40.402630+0000","last_unstale":"2024-04-26T21:09:45.568444+0000","last_undegraded":"2024-04-26T21:09:45.568444+0000","last_fullsized":"2024-04-26T21:09:45.568444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:37.555164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102987+0000","last_change":"2024-04-26T21:09:41.069407+0000","last_active":"2024-04-26T21:09:42.102987+0000","last_peered":"2024-04-26T21:09:42.102987+0000","last_clean":"2024-04-26T21:09:42.102987+0000","last_became_active":"2024-04-26T21:09:41.067522+0000","last_became_peered":"2024-04-26T21:09:41.067522+0000","last_unstale":"2024-04-26T21:09:42.102987+0000","last_undegraded":"2024-04-26T21:09:42.102987+0000","last_fullsized":"2024-04-26T21:09:42.102987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.338665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.576252+0000","last_change":"2024-04-26T21:09:40.391323+0000","last_active":"2024-04-26T21:09:45.576252+0000","last_peered":"2024-04-26T21:09:45.576252+0000","last_clean":"2024-04-26T21:09:45.576252+0000","last_became_active":"2024-04-26T21:09:40.390982+0000","last_became_peered":"2024-04-26T21:09:40.390982+0000","last_unstale":"2024-04-26T21:09:45.576252+0000","last_undegraded":"2024-04-26T21:09:45.576252+0000","last_fullsized":"2024-04-26T21:09:45.576252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:03.465013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101799+0000","last_change":"2024-04-26T21:09:41.068305+0000","last_active":"2024-04-26T21:09:42.101799+0000","last_peered":"2024-04-26T21:09:42.101799+0000","last_clean":"2024-04-26T21:09:42.101799+0000","last_became_active":"2024-04-26T21:09:41.067946+0000","last_became_peered":"2024-04-26T21:09:41.067946+0000","last_unstale":"2024-04-26T21:09:42.101799+0000","last_undegraded":"2024-04-26T21:09:42.101799+0000","last_fullsized":"2024-04-26T21:09:42.101799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:38.761142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.497595+0000","last_change":"2024-04-26T21:09:40.387004+0000","last_active":"2024-04-26T21:09:48.497595+0000","last_peered":"2024-04-26T21:09:48.497595+0000","last_clean":"2024-04-26T21:09:48.497595+0000","last_became_active":"2024-04-26T21:09:40.382385+0000","last_became_peered":"2024-04-26T21:09:40.382385+0000","last_unstale":"2024-04-26T21:09:48.497595+0000","last_undegraded":"2024-04-26T21:09:48.497595+0000","last_fullsized":"2024-04-26T21:09:48.497595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:40.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036681+0000","last_change":"2024-04-26T21:09:41.071024+0000","last_active":"2024-04-26T21:09:42.036681+0000","last_peered":"2024-04-26T21:09:42.036681+0000","last_clean":"2024-04-26T21:09:42.036681+0000","last_became_active":"2024-04-26T21:09:41.070456+0000","last_became_peered":"2024-04-26T21:09:41.070456+0000","last_unstale":"2024-04-26T21:09:42.036681+0000","last_undegraded":"2024-04-26T21:09:42.036681+0000","last_fullsized":"2024-04-26T21:09:42.036681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:56.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.563470+0000","last_change":"2024-04-26T21:09:40.404128+0000","last_active":"2024-04-26T21:09:46.563470+0000","last_peered":"2024-04-26T21:09:46.563470+0000","last_clean":"2024-04-26T21:09:46.563470+0000","last_became_active":"2024-04-26T21:09:40.403760+0000","last_became_peered":"2024-04-26T21:09:40.403760+0000","last_unstale":"2024-04-26T21:09:46.563470+0000","last_undegraded":"2024-04-26T21:09:46.563470+0000","last_fullsized":"2024-04-26T21:09:46.563470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103396+0000","last_change":"2024-04-26T21:09:41.067676+0000","last_active":"2024-04-26T21:09:42.103396+0000","last_peered":"2024-04-26T21:09:42.103396+0000","last_clean":"2024-04-26T21:09:42.103396+0000","last_became_active":"2024-04-26T21:09:41.066736+0000","last_became_peered":"2024-04-26T21:09:41.066736+0000","last_unstale":"2024-04-26T21:09:42.103396+0000","last_undegraded":"2024-04-26T21:09:42.103396+0000","last_fullsized":"2024-04-26T21:09:42.103396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:39.477690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.576797+0000","last_change":"2024-04-26T21:09:40.392845+0000","last_active":"2024-04-26T21:09:45.576797+0000","last_peered":"2024-04-26T21:09:45.576797+0000","last_clean":"2024-04-26T21:09:45.576797+0000","last_became_active":"2024-04-26T21:09:40.392488+0000","last_became_peered":"2024-04-26T21:09:40.392488+0000","last_unstale":"2024-04-26T21:09:45.576797+0000","last_undegraded":"2024-04-26T21:09:45.576797+0000","last_fullsized":"2024-04-26T21:09:45.576797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:54.493294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037601+0000","last_change":"2024-04-26T21:09:41.074270+0000","last_active":"2024-04-26T21:09:42.037601+0000","last_peered":"2024-04-26T21:09:42.037601+0000","last_clean":"2024-04-26T21:09:42.037601+0000","last_became_active":"2024-04-26T21:09:41.073847+0000","last_became_peered":"2024-04-26T21:09:41.073847+0000","last_unstale":"2024-04-26T21:09:42.037601+0000","last_undegraded":"2024-04-26T21:09:42.037601+0000","last_fullsized":"2024-04-26T21:09:42.037601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:03.737883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.606891+0000","last_change":"2024-04-26T21:09:40.397644+0000","last_active":"2024-04-26T21:09:47.606891+0000","last_peered":"2024-04-26T21:09:47.606891+0000","last_clean":"2024-04-26T21:09:47.606891+0000","last_became_active":"2024-04-26T21:09:40.397234+0000","last_became_peered":"2024-04-26T21:09:40.397234+0000","last_unstale":"2024-04-26T21:09:47.606891+0000","last_undegraded":"2024-04-26T21:09:47.606891+0000","last_fullsized":"2024-04-26T21:09:47.606891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:32.220330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038690+0000","last_change":"2024-04-26T21:09:41.072171+0000","last_active":"2024-04-26T21:09:42.038690+0000","last_peered":"2024-04-26T21:09:42.038690+0000","last_clean":"2024-04-26T21:09:42.038690+0000","last_became_active":"2024-04-26T21:09:41.071706+0000","last_became_peered":"2024-04-26T21:09:41.071706+0000","last_unstale":"2024-04-26T21:09:42.038690+0000","last_undegraded":"2024-04-26T21:09:42.038690+0000","last_fullsized":"2024-04-26T21:09:42.038690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:14.286865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105022+0000","last_change":"2024-04-26T21:09:40.393724+0000","last_active":"2024-04-26T21:09:42.105022+0000","last_peered":"2024-04-26T21:09:42.105022+0000","last_clean":"2024-04-26T21:09:42.105022+0000","last_became_active":"2024-04-26T21:09:40.392441+0000","last_became_peered":"2024-04-26T21:09:40.392441+0000","last_unstale":"2024-04-26T21:09:42.105022+0000","last_undegraded":"2024-04-26T21:09:42.105022+0000","last_fullsized":"2024-04-26T21:09:42.105022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:11.989418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037664+0000","last_change":"2024-04-26T21:09:41.071314+0000","last_active":"2024-04-26T21:09:42.037664+0000","last_peered":"2024-04-26T21:09:42.037664+0000","last_clean":"2024-04-26T21:09:42.037664+0000","last_became_active":"2024-04-26T21:09:41.070874+0000","last_became_peered":"2024-04-26T21:09:41.070874+0000","last_unstale":"2024-04-26T21:09:42.037664+0000","last_undegraded":"2024-04-26T21:09:42.037664+0000","last_fullsized":"2024-04-26T21:09:42.037664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:33.877338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.792361+0000","last_change":"2024-04-26T21:09:40.400884+0000","last_active":"2024-04-26T21:09:42.792361+0000","last_peered":"2024-04-26T21:09:42.792361+0000","last_clean":"2024-04-26T21:09:42.792361+0000","last_became_active":"2024-04-26T21:09:40.400326+0000","last_became_peered":"2024-04-26T21:09:40.400326+0000","last_unstale":"2024-04-26T21:09:42.792361+0000","last_undegraded":"2024-04-26T21:09:42.792361+0000","last_fullsized":"2024-04-26T21:09:42.792361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:13.114233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038348+0000","last_change":"2024-04-26T21:09:41.071959+0000","last_active":"2024-04-26T21:09:42.038348+0000","last_peered":"2024-04-26T21:09:42.038348+0000","last_clean":"2024-04-26T21:09:42.038348+0000","last_became_active":"2024-04-26T21:09:41.071601+0000","last_became_peered":"2024-04-26T21:09:41.071601+0000","last_unstale":"2024-04-26T21:09:42.038348+0000","last_undegraded":"2024-04-26T21:09:42.038348+0000","last_fullsized":"2024-04-26T21:09:42.038348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:29.806956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036534+0000","last_change":"2024-04-26T21:09:41.066764+0000","last_active":"2024-04-26T21:09:42.036534+0000","last_peered":"2024-04-26T21:09:42.036534+0000","last_clean":"2024-04-26T21:09:42.036534+0000","last_became_active":"2024-04-26T21:09:41.066333+0000","last_became_peered":"2024-04-26T21:09:41.066333+0000","last_unstale":"2024-04-26T21:09:42.036534+0000","last_undegraded":"2024-04-26T21:09:42.036534+0000","last_fullsized":"2024-04-26T21:09:42.036534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:32.680215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.099875+0000","last_change":"2024-04-26T21:09:41.072567+0000","last_active":"2024-04-26T21:09:42.099875+0000","last_peered":"2024-04-26T21:09:42.099875+0000","last_clean":"2024-04-26T21:09:42.099875+0000","last_became_active":"2024-04-26T21:09:41.072033+0000","last_became_peered":"2024-04-26T21:09:41.072033+0000","last_unstale":"2024-04-26T21:09:42.099875+0000","last_undegraded":"2024-04-26T21:09:42.099875+0000","last_fullsized":"2024-04-26T21:09:42.099875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:37.178970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100637+0000","last_change":"2024-04-26T21:09:41.071111+0000","last_active":"2024-04-26T21:09:42.100637+0000","last_peered":"2024-04-26T21:09:42.100637+0000","last_clean":"2024-04-26T21:09:42.100637+0000","last_became_active":"2024-04-26T21:09:41.070354+0000","last_became_peered":"2024-04-26T21:09:41.070354+0000","last_unstale":"2024-04-26T21:09:42.100637+0000","last_undegraded":"2024-04-26T21:09:42.100637+0000","last_fullsized":"2024-04-26T21:09:42.100637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:10.990058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101560+0000","last_change":"2024-04-26T21:09:41.076927+0000","last_active":"2024-04-26T21:09:42.101560+0000","last_peered":"2024-04-26T21:09:42.101560+0000","last_clean":"2024-04-26T21:09:42.101560+0000","last_became_active":"2024-04-26T21:09:41.076600+0000","last_became_peered":"2024-04-26T21:09:41.076600+0000","last_unstale":"2024-04-26T21:09:42.101560+0000","last_undegraded":"2024-04-26T21:09:42.101560+0000","last_fullsized":"2024-04-26T21:09:42.101560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:25.960409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104838+0000","last_change":"2024-04-26T21:09:41.079293+0000","last_active":"2024-04-26T21:09:42.104838+0000","last_peered":"2024-04-26T21:09:42.104838+0000","last_clean":"2024-04-26T21:09:42.104838+0000","last_became_active":"2024-04-26T21:09:41.077473+0000","last_became_peered":"2024-04-26T21:09:41.077473+0000","last_unstale":"2024-04-26T21:09:42.104838+0000","last_undegraded":"2024-04-26T21:09:42.104838+0000","last_fullsized":"2024-04-26T21:09:42.104838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:47.281535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036886+0000","last_change":"2024-04-26T21:09:41.065914+0000","last_active":"2024-04-26T21:09:42.036886+0000","last_peered":"2024-04-26T21:09:42.036886+0000","last_clean":"2024-04-26T21:09:42.036886+0000","last_became_active":"2024-04-26T21:09:41.065477+0000","last_became_peered":"2024-04-26T21:09:41.065477+0000","last_unstale":"2024-04-26T21:09:42.036886+0000","last_undegraded":"2024-04-26T21:09:42.036886+0000","last_fullsized":"2024-04-26T21:09:42.036886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:40.517174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103823+0000","last_change":"2024-04-26T21:09:41.068594+0000","last_active":"2024-04-26T21:09:42.103823+0000","last_peered":"2024-04-26T21:09:42.103823+0000","last_clean":"2024-04-26T21:09:42.103823+0000","last_became_active":"2024-04-26T21:09:41.068238+0000","last_became_peered":"2024-04-26T21:09:41.068238+0000","last_unstale":"2024-04-26T21:09:42.103823+0000","last_undegraded":"2024-04-26T21:09:42.103823+0000","last_fullsized":"2024-04-26T21:09:42.103823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:58.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101141+0000","last_change":"2024-04-26T21:09:41.081684+0000","last_active":"2024-04-26T21:09:42.101141+0000","last_peered":"2024-04-26T21:09:42.101141+0000","last_clean":"2024-04-26T21:09:42.101141+0000","last_became_active":"2024-04-26T21:09:41.074963+0000","last_became_peered":"2024-04-26T21:09:41.074963+0000","last_unstale":"2024-04-26T21:09:42.101141+0000","last_undegraded":"2024-04-26T21:09:42.101141+0000","last_fullsized":"2024-04-26T21:09:42.101141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:14.736390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102778+0000","last_change":"2024-04-26T21:09:41.068052+0000","last_active":"2024-04-26T21:09:42.102778+0000","last_peered":"2024-04-26T21:09:42.102778+0000","last_clean":"2024-04-26T21:09:42.102778+0000","last_became_active":"2024-04-26T21:09:41.067728+0000","last_became_peered":"2024-04-26T21:09:41.067728+0000","last_unstale":"2024-04-26T21:09:42.102778+0000","last_undegraded":"2024-04-26T21:09:42.102778+0000","last_fullsized":"2024-04-26T21:09:42.102778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:19.186963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100313+0000","last_change":"2024-04-26T21:09:41.078725+0000","last_active":"2024-04-26T21:09:42.100313+0000","last_peered":"2024-04-26T21:09:42.100313+0000","last_clean":"2024-04-26T21:09:42.100313+0000","last_became_active":"2024-04-26T21:09:41.075757+0000","last_became_peered":"2024-04-26T21:09:41.075757+0000","last_unstale":"2024-04-26T21:09:42.100313+0000","last_undegraded":"2024-04-26T21:09:42.100313+0000","last_fullsized":"2024-04-26T21:09:42.100313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:50.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102664+0000","last_change":"2024-04-26T21:09:41.067872+0000","last_active":"2024-04-26T21:09:42.102664+0000","last_peered":"2024-04-26T21:09:42.102664+0000","last_clean":"2024-04-26T21:09:42.102664+0000","last_became_active":"2024-04-26T21:09:41.067476+0000","last_became_peered":"2024-04-26T21:09:41.067476+0000","last_unstale":"2024-04-26T21:09:42.102664+0000","last_undegraded":"2024-04-26T21:09:42.102664+0000","last_fullsized":"2024-04-26T21:09:42.102664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:54.740719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105327+0000","last_change":"2024-04-26T21:09:41.080571+0000","last_active":"2024-04-26T21:09:42.105327+0000","last_peered":"2024-04-26T21:09:42.105327+0000","last_clean":"2024-04-26T21:09:42.105327+0000","last_became_active":"2024-04-26T21:09:41.078465+0000","last_became_peered":"2024-04-26T21:09:41.078465+0000","last_unstale":"2024-04-26T21:09:42.105327+0000","last_undegraded":"2024-04-26T21:09:42.105327+0000","last_fullsized":"2024-04-26T21:09:42.105327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:27.636693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036697+0000","last_change":"2024-04-26T21:09:41.080111+0000","last_active":"2024-04-26T21:09:42.036697+0000","last_peered":"2024-04-26T21:09:42.036697+0000","last_clean":"2024-04-26T21:09:42.036697+0000","last_became_active":"2024-04-26T21:09:41.079220+0000","last_became_peered":"2024-04-26T21:09:41.079220+0000","last_unstale":"2024-04-26T21:09:42.036697+0000","last_undegraded":"2024-04-26T21:09:42.036697+0000","last_fullsized":"2024-04-26T21:09:42.036697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.055957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4618,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":35767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":453017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547223,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1045371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710750,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":453343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972390,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":453039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266732,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1044007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":452525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":475082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":455397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":452843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":453079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1552384,"data_stored":1042985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:09:51.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:09:51.888 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:51 smithi027 ceph-mon[28085]: pgmap v372: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 113 B/s rd, 3.4 KiB/s wr, 13 op/s 2024-04-26T21:09:51.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:51 smithi027 ceph-mon[28085]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:51.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/82737170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:51.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1284442174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:51 smithi204 ceph-mon[32145]: pgmap v372: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 113 B/s rd, 3.4 KiB/s wr, 13 op/s 2024-04-26T21:09:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:51 smithi204 ceph-mon[32145]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/82737170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:52.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1284442174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-mon[31273]: pgmap v372: 97 pgs: 97 active+clean; 581 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 113 B/s rd, 3.4 KiB/s wr, 13 op/s 2024-04-26T21:09:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-mon[31273]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:09:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/82737170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:09:52.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1284442174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 -- 172.21.15.27:0/2709130502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109f80 msgr2=0x7f964010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 --2- 172.21.15.27:0/2709130502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109f80 0x7f964010a400 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9628009900 tx=0x7f962802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 -- 172.21.15.27:0/2709130502 shutdown_connections 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 --2- 172.21.15.27:0/2709130502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 0x7f96401111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 --2- 172.21.15.27:0/2709130502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640109f80 0x7f964010a400 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 --2- 172.21.15.27:0/2709130502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640108d90 0x7f96401091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 -- 172.21.15.27:0/2709130502 >> 172.21.15.27:0/2709130502 conn(0x7f9640077d30 msgr2=0x7f964007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.030+0000 7f963f200640 1 -- 172.21.15.27:0/2709130502 shutdown_connections 2024-04-26T21:09:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.031+0000 7f963f200640 1 -- 172.21.15.27:0/2709130502 wait complete. 2024-04-26T21:09:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.031+0000 7f963f200640 1 Processor -- start 2024-04-26T21:09:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 -- start start 2024-04-26T21:09:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640109f80 0x7f964019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 0x7f96401a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96401147c0 con 0x7f9640108d90 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9640114640 con 0x7f9640109f80 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9640114940 con 0x7f964010a940 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36360/0 (socket says 172.21.15.27:36360) 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.032+0000 7f963de00640 1 -- 172.21.15.27:0/1642477620 learned_addr learned my addr 172.21.15.27:0/1642477620 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963d400640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640109f80 0x7f964019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963e800640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 0x7f96401a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 -- 172.21.15.27:0/1642477620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 msgr2=0x7f96401a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 0x7f96401a6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 -- 172.21.15.27:0/1642477620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640109f80 msgr2=0x7f964019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640109f80 0x7f964019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 -- 172.21.15.27:0/1642477620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401a6ab0 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963de00640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f963000d770 tx=0x7f963000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630017070 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401a6da0 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9630009c30 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630003c50 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.033+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a7200 con 0x7f9640108d90 2024-04-26T21:09:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.035+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640110180 con 0x7f9640108d90 2024-04-26T21:09:52.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.037+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9630003df0 con 0x7f9640108d90 2024-04-26T21:09:52.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.038+0000 7f9636a00640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 0x7f96240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.039+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f96300cff70 con 0x7f9640108d90 2024-04-26T21:09:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.039+0000 7f963d400640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 0x7f96240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.040+0000 7f963d400640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 0x7f96240a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f9628031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.043+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f963009c8a0 con 0x7f9640108d90 2024-04-26T21:09:52.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.248+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9640062d70 con 0x7f96240a3c40 2024-04-26T21:09:52.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.273+0000 7f9636a00640 1 -- 172.21.15.27:0/1642477620 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258897 (secure 0 0 0) 0x7f9640062d70 con 0x7f96240a3c40 2024-04-26T21:09:52.278 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:52.282 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T21:09:52.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.278+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 msgr2=0x7f96240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.278+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 0x7f96240a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f9628031400 comp rx=0 tx=0).stop 2024-04-26T21:09:52.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.278+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 msgr2=0x7f964019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.278+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f963000d770 tx=0x7f963000dc30 comp rx=0 tx=0).stop 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 shutdown_connections 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96240a3c40 0x7f96240a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f964010a940 0x7f96401a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9640109f80 0x7f964019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 --2- 172.21.15.27:0/1642477620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9640108d90 0x7f964019ee90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 >> 172.21.15.27:0/1642477620 conn(0x7f9640077d30 msgr2=0x7f964010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 shutdown_connections 2024-04-26T21:09:52.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:52.279+0000 7f963f200640 1 -- 172.21.15.27:0/1642477620 wait complete. 2024-04-26T21:09:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-26T21:09:52.046053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641620,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":185,"num_read_kb":194,"num_write":393,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":62,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":325,"ondisk_log_size":325,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395648,"kb_used_data":13216,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124546176,"statfs":{"total":1151940427776,"available":1151535284224,"internally_reserved":0,"allocated":13533184,"data_stored":7280332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51252,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":49,"num_read_kb":75,"num_write":209,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.014110"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037642+0000","last_change":"2024-04-26T21:09:41.072073+0000","last_active":"2024-04-26T21:09:42.037642+0000","last_peered":"2024-04-26T21:09:42.037642+0000","last_clean":"2024-04-26T21:09:42.037642+0000","last_became_active":"2024-04-26T21:09:41.071687+0000","last_became_peered":"2024-04-26T21:09:41.071687+0000","last_unstale":"2024-04-26T21:09:42.037642+0000","last_undegraded":"2024-04-26T21:09:42.037642+0000","last_fullsized":"2024-04-26T21:09:42.037642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:17.092134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104814+0000","last_change":"2024-04-26T21:09:41.063568+0000","last_active":"2024-04-26T21:09:42.104814+0000","last_peered":"2024-04-26T21:09:42.104814+0000","last_clean":"2024-04-26T21:09:42.104814+0000","last_became_active":"2024-04-26T21:09:41.061712+0000","last_became_peered":"2024-04-26T21:09:41.061712+0000","last_unstale":"2024-04-26T21:09:42.104814+0000","last_undegraded":"2024-04-26T21:09:42.104814+0000","last_fullsized":"2024-04-26T21:09:42.104814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:38.894168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103731+0000","last_change":"2024-04-26T21:09:41.068164+0000","last_active":"2024-04-26T21:09:42.103731+0000","last_peered":"2024-04-26T21:09:42.103731+0000","last_clean":"2024-04-26T21:09:42.103731+0000","last_became_active":"2024-04-26T21:09:41.067074+0000","last_became_peered":"2024-04-26T21:09:41.067074+0000","last_unstale":"2024-04-26T21:09:42.103731+0000","last_undegraded":"2024-04-26T21:09:42.103731+0000","last_fullsized":"2024-04-26T21:09:42.103731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.900193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101822+0000","last_change":"2024-04-26T21:09:41.076541+0000","last_active":"2024-04-26T21:09:42.101822+0000","last_peered":"2024-04-26T21:09:42.101822+0000","last_clean":"2024-04-26T21:09:42.101822+0000","last_became_active":"2024-04-26T21:09:41.074492+0000","last_became_peered":"2024-04-26T21:09:41.074492+0000","last_unstale":"2024-04-26T21:09:42.101822+0000","last_undegraded":"2024-04-26T21:09:42.101822+0000","last_fullsized":"2024-04-26T21:09:42.101822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:56.007862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103202+0000","last_change":"2024-04-26T21:09:41.070626+0000","last_active":"2024-04-26T21:09:42.103202+0000","last_peered":"2024-04-26T21:09:42.103202+0000","last_clean":"2024-04-26T21:09:42.103202+0000","last_became_active":"2024-04-26T21:09:41.070258+0000","last_became_peered":"2024-04-26T21:09:41.070258+0000","last_unstale":"2024-04-26T21:09:42.103202+0000","last_undegraded":"2024-04-26T21:09:42.103202+0000","last_fullsized":"2024-04-26T21:09:42.103202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:10.491491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104645+0000","last_change":"2024-04-26T21:09:41.066628+0000","last_active":"2024-04-26T21:09:42.104645+0000","last_peered":"2024-04-26T21:09:42.104645+0000","last_clean":"2024-04-26T21:09:42.104645+0000","last_became_active":"2024-04-26T21:09:41.066217+0000","last_became_peered":"2024-04-26T21:09:41.066217+0000","last_unstale":"2024-04-26T21:09:42.104645+0000","last_undegraded":"2024-04-26T21:09:42.104645+0000","last_fullsized":"2024-04-26T21:09:42.104645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:10:27.575166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037031+0000","last_change":"2024-04-26T21:09:41.069684+0000","last_active":"2024-04-26T21:09:42.037031+0000","last_peered":"2024-04-26T21:09:42.037031+0000","last_clean":"2024-04-26T21:09:42.037031+0000","last_became_active":"2024-04-26T21:09:41.069323+0000","last_became_peered":"2024-04-26T21:09:41.069323+0000","last_unstale":"2024-04-26T21:09:42.037031+0000","last_undegraded":"2024-04-26T21:09:42.037031+0000","last_fullsized":"2024-04-26T21:09:42.037031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.122606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100229+0000","last_change":"2024-04-26T21:09:41.069130+0000","last_active":"2024-04-26T21:09:42.100229+0000","last_peered":"2024-04-26T21:09:42.100229+0000","last_clean":"2024-04-26T21:09:42.100229+0000","last_became_active":"2024-04-26T21:09:41.068682+0000","last_became_peered":"2024-04-26T21:09:41.068682+0000","last_unstale":"2024-04-26T21:09:42.100229+0000","last_undegraded":"2024-04-26T21:09:42.100229+0000","last_fullsized":"2024-04-26T21:09:42.100229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:39.389562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102028+0000","last_change":"2024-04-26T21:09:41.073930+0000","last_active":"2024-04-26T21:09:42.102028+0000","last_peered":"2024-04-26T21:09:42.102028+0000","last_clean":"2024-04-26T21:09:42.102028+0000","last_became_active":"2024-04-26T21:09:41.072566+0000","last_became_peered":"2024-04-26T21:09:41.072566+0000","last_unstale":"2024-04-26T21:09:42.102028+0000","last_undegraded":"2024-04-26T21:09:42.102028+0000","last_fullsized":"2024-04-26T21:09:42.102028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:48.613310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105196+0000","last_change":"2024-04-26T21:09:41.065598+0000","last_active":"2024-04-26T21:09:42.105196+0000","last_peered":"2024-04-26T21:09:42.105196+0000","last_clean":"2024-04-26T21:09:42.105196+0000","last_became_active":"2024-04-26T21:09:41.064977+0000","last_became_peered":"2024-04-26T21:09:41.064977+0000","last_unstale":"2024-04-26T21:09:42.105196+0000","last_undegraded":"2024-04-26T21:09:42.105196+0000","last_fullsized":"2024-04-26T21:09:42.105196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:39.776967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038255+0000","last_change":"2024-04-26T21:09:41.077939+0000","last_active":"2024-04-26T21:09:42.038255+0000","last_peered":"2024-04-26T21:09:42.038255+0000","last_clean":"2024-04-26T21:09:42.038255+0000","last_became_active":"2024-04-26T21:09:41.077641+0000","last_became_peered":"2024-04-26T21:09:41.077641+0000","last_unstale":"2024-04-26T21:09:42.038255+0000","last_undegraded":"2024-04-26T21:09:42.038255+0000","last_fullsized":"2024-04-26T21:09:42.038255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:47.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101331+0000","last_change":"2024-04-26T21:09:41.075109+0000","last_active":"2024-04-26T21:09:42.101331+0000","last_peered":"2024-04-26T21:09:42.101331+0000","last_clean":"2024-04-26T21:09:42.101331+0000","last_became_active":"2024-04-26T21:09:41.073866+0000","last_became_peered":"2024-04-26T21:09:41.073866+0000","last_unstale":"2024-04-26T21:09:42.101331+0000","last_undegraded":"2024-04-26T21:09:42.101331+0000","last_fullsized":"2024-04-26T21:09:42.101331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:16.786155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105287+0000","last_change":"2024-04-26T21:09:41.074934+0000","last_active":"2024-04-26T21:09:42.105287+0000","last_peered":"2024-04-26T21:09:42.105287+0000","last_clean":"2024-04-26T21:09:42.105287+0000","last_became_active":"2024-04-26T21:09:41.072388+0000","last_became_peered":"2024-04-26T21:09:41.072388+0000","last_unstale":"2024-04-26T21:09:42.105287+0000","last_undegraded":"2024-04-26T21:09:42.105287+0000","last_fullsized":"2024-04-26T21:09:42.105287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:23.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100481+0000","last_change":"2024-04-26T21:09:41.062598+0000","last_active":"2024-04-26T21:09:42.100481+0000","last_peered":"2024-04-26T21:09:42.100481+0000","last_clean":"2024-04-26T21:09:42.100481+0000","last_became_active":"2024-04-26T21:09:41.061828+0000","last_became_peered":"2024-04-26T21:09:41.061828+0000","last_unstale":"2024-04-26T21:09:42.100481+0000","last_undegraded":"2024-04-26T21:09:42.100481+0000","last_fullsized":"2024-04-26T21:09:42.100481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:52.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100659+0000","last_change":"2024-04-26T21:09:41.064874+0000","last_active":"2024-04-26T21:09:42.100659+0000","last_peered":"2024-04-26T21:09:42.100659+0000","last_clean":"2024-04-26T21:09:42.100659+0000","last_became_active":"2024-04-26T21:09:41.060190+0000","last_became_peered":"2024-04-26T21:09:41.060190+0000","last_unstale":"2024-04-26T21:09:42.100659+0000","last_undegraded":"2024-04-26T21:09:42.100659+0000","last_fullsized":"2024-04-26T21:09:42.100659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101169+0000","last_change":"2024-04-26T21:09:41.067343+0000","last_active":"2024-04-26T21:09:42.101169+0000","last_peered":"2024-04-26T21:09:42.101169+0000","last_clean":"2024-04-26T21:09:42.101169+0000","last_became_active":"2024-04-26T21:09:41.066248+0000","last_became_peered":"2024-04-26T21:09:41.066248+0000","last_unstale":"2024-04-26T21:09:42.101169+0000","last_undegraded":"2024-04-26T21:09:42.101169+0000","last_fullsized":"2024-04-26T21:09:42.101169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:40.476089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038084+0000","last_change":"2024-04-26T21:09:41.079414+0000","last_active":"2024-04-26T21:09:42.038084+0000","last_peered":"2024-04-26T21:09:42.038084+0000","last_clean":"2024-04-26T21:09:42.038084+0000","last_became_active":"2024-04-26T21:09:41.078949+0000","last_became_peered":"2024-04-26T21:09:41.078949+0000","last_unstale":"2024-04-26T21:09:42.038084+0000","last_undegraded":"2024-04-26T21:09:42.038084+0000","last_fullsized":"2024-04-26T21:09:42.038084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:19.947290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036239+0000","last_change":"2024-04-26T21:09:41.069376+0000","last_active":"2024-04-26T21:09:42.036239+0000","last_peered":"2024-04-26T21:09:42.036239+0000","last_clean":"2024-04-26T21:09:42.036239+0000","last_became_active":"2024-04-26T21:09:41.069049+0000","last_became_peered":"2024-04-26T21:09:41.069049+0000","last_unstale":"2024-04-26T21:09:42.036239+0000","last_undegraded":"2024-04-26T21:09:42.036239+0000","last_fullsized":"2024-04-26T21:09:42.036239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:25.058292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104058+0000","last_change":"2024-04-26T21:09:41.072485+0000","last_active":"2024-04-26T21:09:42.104058+0000","last_peered":"2024-04-26T21:09:42.104058+0000","last_clean":"2024-04-26T21:09:42.104058+0000","last_became_active":"2024-04-26T21:09:41.070990+0000","last_became_peered":"2024-04-26T21:09:41.070990+0000","last_unstale":"2024-04-26T21:09:42.104058+0000","last_undegraded":"2024-04-26T21:09:42.104058+0000","last_fullsized":"2024-04-26T21:09:42.104058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:40.648097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.501095+0000","last_change":"2024-04-26T21:09:40.391208+0000","last_active":"2024-04-26T21:09:48.501095+0000","last_peered":"2024-04-26T21:09:48.501095+0000","last_clean":"2024-04-26T21:09:48.501095+0000","last_became_active":"2024-04-26T21:09:40.390216+0000","last_became_peered":"2024-04-26T21:09:40.390216+0000","last_unstale":"2024-04-26T21:09:48.501095+0000","last_undegraded":"2024-04-26T21:09:48.501095+0000","last_fullsized":"2024-04-26T21:09:48.501095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:40.516383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100509+0000","last_change":"2024-04-26T21:09:41.084249+0000","last_active":"2024-04-26T21:09:42.100509+0000","last_peered":"2024-04-26T21:09:42.100509+0000","last_clean":"2024-04-26T21:09:42.100509+0000","last_became_active":"2024-04-26T21:09:41.083580+0000","last_became_peered":"2024-04-26T21:09:41.083580+0000","last_unstale":"2024-04-26T21:09:42.100509+0000","last_undegraded":"2024-04-26T21:09:42.100509+0000","last_fullsized":"2024-04-26T21:09:42.100509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:02.615403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:43.526770+0000","last_change":"2024-04-26T21:09:40.389376+0000","last_active":"2024-04-26T21:09:43.526770+0000","last_peered":"2024-04-26T21:09:43.526770+0000","last_clean":"2024-04-26T21:09:43.526770+0000","last_became_active":"2024-04-26T21:09:40.388761+0000","last_became_peered":"2024-04-26T21:09:40.388761+0000","last_unstale":"2024-04-26T21:09:43.526770+0000","last_undegraded":"2024-04-26T21:09:43.526770+0000","last_fullsized":"2024-04-26T21:09:43.526770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:39.940524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102397+0000","last_change":"2024-04-26T21:09:41.073193+0000","last_active":"2024-04-26T21:09:42.102397+0000","last_peered":"2024-04-26T21:09:42.102397+0000","last_clean":"2024-04-26T21:09:42.102397+0000","last_became_active":"2024-04-26T21:09:41.071569+0000","last_became_peered":"2024-04-26T21:09:41.071569+0000","last_unstale":"2024-04-26T21:09:42.102397+0000","last_undegraded":"2024-04-26T21:09:42.102397+0000","last_fullsized":"2024-04-26T21:09:42.102397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:40.078386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.499135+0000","last_change":"2024-04-26T21:09:40.394188+0000","last_active":"2024-04-26T21:09:48.499135+0000","last_peered":"2024-04-26T21:09:48.499135+0000","last_clean":"2024-04-26T21:09:48.499135+0000","last_became_active":"2024-04-26T21:09:40.393854+0000","last_became_peered":"2024-04-26T21:09:40.393854+0000","last_unstale":"2024-04-26T21:09:48.499135+0000","last_undegraded":"2024-04-26T21:09:48.499135+0000","last_fullsized":"2024-04-26T21:09:48.499135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:30.232353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103098+0000","last_change":"2024-04-26T21:09:41.068545+0000","last_active":"2024-04-26T21:09:42.103098+0000","last_peered":"2024-04-26T21:09:42.103098+0000","last_clean":"2024-04-26T21:09:42.103098+0000","last_became_active":"2024-04-26T21:09:41.066805+0000","last_became_peered":"2024-04-26T21:09:41.066805+0000","last_unstale":"2024-04-26T21:09:42.103098+0000","last_undegraded":"2024-04-26T21:09:42.103098+0000","last_fullsized":"2024-04-26T21:09:42.103098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:32.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.496047+0000","last_change":"2024-04-26T21:09:40.382727+0000","last_active":"2024-04-26T21:09:48.496047+0000","last_peered":"2024-04-26T21:09:48.496047+0000","last_clean":"2024-04-26T21:09:48.496047+0000","last_became_active":"2024-04-26T21:09:40.380205+0000","last_became_peered":"2024-04-26T21:09:40.380205+0000","last_unstale":"2024-04-26T21:09:48.496047+0000","last_undegraded":"2024-04-26T21:09:48.496047+0000","last_fullsized":"2024-04-26T21:09:48.496047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:32.981478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105299+0000","last_change":"2024-04-26T21:09:41.075664+0000","last_active":"2024-04-26T21:09:42.105299+0000","last_peered":"2024-04-26T21:09:42.105299+0000","last_clean":"2024-04-26T21:09:42.105299+0000","last_became_active":"2024-04-26T21:09:41.075213+0000","last_became_peered":"2024-04-26T21:09:41.075213+0000","last_unstale":"2024-04-26T21:09:42.105299+0000","last_undegraded":"2024-04-26T21:09:42.105299+0000","last_fullsized":"2024-04-26T21:09:42.105299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:11.832226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.486869+0000","last_change":"2024-04-26T21:09:40.395282+0000","last_active":"2024-04-26T21:09:48.486869+0000","last_peered":"2024-04-26T21:09:48.486869+0000","last_clean":"2024-04-26T21:09:48.486869+0000","last_became_active":"2024-04-26T21:09:40.394858+0000","last_became_peered":"2024-04-26T21:09:40.394858+0000","last_unstale":"2024-04-26T21:09:48.486869+0000","last_undegraded":"2024-04-26T21:09:48.486869+0000","last_fullsized":"2024-04-26T21:09:48.486869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:38.330771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102517+0000","last_change":"2024-04-26T21:09:41.064853+0000","last_active":"2024-04-26T21:09:42.102517+0000","last_peered":"2024-04-26T21:09:42.102517+0000","last_clean":"2024-04-26T21:09:42.102517+0000","last_became_active":"2024-04-26T21:09:41.064236+0000","last_became_peered":"2024-04-26T21:09:41.064236+0000","last_unstale":"2024-04-26T21:09:42.102517+0000","last_undegraded":"2024-04-26T21:09:42.102517+0000","last_fullsized":"2024-04-26T21:09:42.102517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:05.549027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.602260+0000","last_change":"2024-04-26T21:09:40.390857+0000","last_active":"2024-04-26T21:09:47.602260+0000","last_peered":"2024-04-26T21:09:47.602260+0000","last_clean":"2024-04-26T21:09:47.602260+0000","last_became_active":"2024-04-26T21:09:40.390513+0000","last_became_peered":"2024-04-26T21:09:40.390513+0000","last_unstale":"2024-04-26T21:09:47.602260+0000","last_undegraded":"2024-04-26T21:09:47.602260+0000","last_fullsized":"2024-04-26T21:09:47.602260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.106177+0000","last_change":"2024-04-26T21:09:41.074105+0000","last_active":"2024-04-26T21:09:42.106177+0000","last_peered":"2024-04-26T21:09:42.106177+0000","last_clean":"2024-04-26T21:09:42.106177+0000","last_became_active":"2024-04-26T21:09:41.073645+0000","last_became_peered":"2024-04-26T21:09:41.073645+0000","last_unstale":"2024-04-26T21:09:42.106177+0000","last_undegraded":"2024-04-26T21:09:42.106177+0000","last_fullsized":"2024-04-26T21:09:42.106177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:04.118506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.585807+0000","last_change":"2024-04-26T21:09:40.390893+0000","last_active":"2024-04-26T21:09:46.585807+0000","last_peered":"2024-04-26T21:09:46.585807+0000","last_clean":"2024-04-26T21:09:46.585807+0000","last_became_active":"2024-04-26T21:09:40.390519+0000","last_became_peered":"2024-04-26T21:09:40.390519+0000","last_unstale":"2024-04-26T21:09:46.585807+0000","last_undegraded":"2024-04-26T21:09:46.585807+0000","last_fullsized":"2024-04-26T21:09:46.585807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:40.963614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101057+0000","last_change":"2024-04-26T21:09:41.074751+0000","last_active":"2024-04-26T21:09:42.101057+0000","last_peered":"2024-04-26T21:09:42.101057+0000","last_clean":"2024-04-26T21:09:42.101057+0000","last_became_active":"2024-04-26T21:09:41.074330+0000","last_became_peered":"2024-04-26T21:09:41.074330+0000","last_unstale":"2024-04-26T21:09:42.101057+0000","last_undegraded":"2024-04-26T21:09:42.101057+0000","last_fullsized":"2024-04-26T21:09:42.101057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:47.759543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.487210+0000","last_change":"2024-04-26T21:09:40.389971+0000","last_active":"2024-04-26T21:09:48.487210+0000","last_peered":"2024-04-26T21:09:48.487210+0000","last_clean":"2024-04-26T21:09:48.487210+0000","last_became_active":"2024-04-26T21:09:40.389093+0000","last_became_peered":"2024-04-26T21:09:40.389093+0000","last_unstale":"2024-04-26T21:09:48.487210+0000","last_undegraded":"2024-04-26T21:09:48.487210+0000","last_fullsized":"2024-04-26T21:09:48.487210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:18.838994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.035164+0000","last_change":"2024-04-26T21:09:41.068903+0000","last_active":"2024-04-26T21:09:42.035164+0000","last_peered":"2024-04-26T21:09:42.035164+0000","last_clean":"2024-04-26T21:09:42.035164+0000","last_became_active":"2024-04-26T21:09:41.068585+0000","last_became_peered":"2024-04-26T21:09:41.068585+0000","last_unstale":"2024-04-26T21:09:42.035164+0000","last_undegraded":"2024-04-26T21:09:42.035164+0000","last_fullsized":"2024-04-26T21:09:42.035164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:18.372213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.491417+0000","last_change":"2024-04-26T21:09:40.391501+0000","last_active":"2024-04-26T21:09:50.491417+0000","last_peered":"2024-04-26T21:09:50.491417+0000","last_clean":"2024-04-26T21:09:50.491417+0000","last_became_active":"2024-04-26T21:09:40.390434+0000","last_became_peered":"2024-04-26T21:09:40.390434+0000","last_unstale":"2024-04-26T21:09:50.491417+0000","last_undegraded":"2024-04-26T21:09:50.491417+0000","last_fullsized":"2024-04-26T21:09:50.491417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:11.854716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102675+0000","last_change":"2024-04-26T21:09:41.064937+0000","last_active":"2024-04-26T21:09:42.102675+0000","last_peered":"2024-04-26T21:09:42.102675+0000","last_clean":"2024-04-26T21:09:42.102675+0000","last_became_active":"2024-04-26T21:09:41.063667+0000","last_became_peered":"2024-04-26T21:09:41.063667+0000","last_unstale":"2024-04-26T21:09:42.102675+0000","last_undegraded":"2024-04-26T21:09:42.102675+0000","last_fullsized":"2024-04-26T21:09:42.102675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.380869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.498423+0000","last_change":"2024-04-26T21:09:40.394499+0000","last_active":"2024-04-26T21:09:48.498423+0000","last_peered":"2024-04-26T21:09:48.498423+0000","last_clean":"2024-04-26T21:09:48.498423+0000","last_became_active":"2024-04-26T21:09:40.394052+0000","last_became_peered":"2024-04-26T21:09:40.394052+0000","last_unstale":"2024-04-26T21:09:48.498423+0000","last_undegraded":"2024-04-26T21:09:48.498423+0000","last_fullsized":"2024-04-26T21:09:48.498423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:51.540913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102187+0000","last_change":"2024-04-26T21:09:41.064559+0000","last_active":"2024-04-26T21:09:42.102187+0000","last_peered":"2024-04-26T21:09:42.102187+0000","last_clean":"2024-04-26T21:09:42.102187+0000","last_became_active":"2024-04-26T21:09:41.063117+0000","last_became_peered":"2024-04-26T21:09:41.063117+0000","last_unstale":"2024-04-26T21:09:42.102187+0000","last_undegraded":"2024-04-26T21:09:42.102187+0000","last_fullsized":"2024-04-26T21:09:42.102187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:34.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.604578+0000","last_change":"2024-04-26T21:09:40.390507+0000","last_active":"2024-04-26T21:09:47.604578+0000","last_peered":"2024-04-26T21:09:47.604578+0000","last_clean":"2024-04-26T21:09:47.604578+0000","last_became_active":"2024-04-26T21:09:40.388503+0000","last_became_peered":"2024-04-26T21:09:40.388503+0000","last_unstale":"2024-04-26T21:09:47.604578+0000","last_undegraded":"2024-04-26T21:09:47.604578+0000","last_fullsized":"2024-04-26T21:09:47.604578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:40.432667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102932+0000","last_change":"2024-04-26T21:09:41.065188+0000","last_active":"2024-04-26T21:09:42.102932+0000","last_peered":"2024-04-26T21:09:42.102932+0000","last_clean":"2024-04-26T21:09:42.102932+0000","last_became_active":"2024-04-26T21:09:41.061218+0000","last_became_peered":"2024-04-26T21:09:41.061218+0000","last_unstale":"2024-04-26T21:09:42.102932+0000","last_undegraded":"2024-04-26T21:09:42.102932+0000","last_fullsized":"2024-04-26T21:09:42.102932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:42.087561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'22","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.504898+0000","last_change":"2024-04-26T21:09:40.393372+0000","last_active":"2024-04-26T21:09:50.504898+0000","last_peered":"2024-04-26T21:09:50.504898+0000","last_clean":"2024-04-26T21:09:50.504898+0000","last_became_active":"2024-04-26T21:09:40.392631+0000","last_became_peered":"2024-04-26T21:09:40.392631+0000","last_unstale":"2024-04-26T21:09:50.504898+0000","last_undegraded":"2024-04-26T21:09:50.504898+0000","last_fullsized":"2024-04-26T21:09:50.504898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.347781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44227,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":44,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102442+0000","last_change":"2024-04-26T21:09:41.063711+0000","last_active":"2024-04-26T21:09:42.102442+0000","last_peered":"2024-04-26T21:09:42.102442+0000","last_clean":"2024-04-26T21:09:42.102442+0000","last_became_active":"2024-04-26T21:09:41.063330+0000","last_became_peered":"2024-04-26T21:09:41.063330+0000","last_unstale":"2024-04-26T21:09:42.102442+0000","last_undegraded":"2024-04-26T21:09:42.102442+0000","last_fullsized":"2024-04-26T21:09:42.102442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:28.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.589498+0000","last_change":"2024-04-26T21:09:40.404407+0000","last_active":"2024-04-26T21:09:46.589498+0000","last_peered":"2024-04-26T21:09:46.589498+0000","last_clean":"2024-04-26T21:09:46.589498+0000","last_became_active":"2024-04-26T21:09:40.402751+0000","last_became_peered":"2024-04-26T21:09:40.402751+0000","last_unstale":"2024-04-26T21:09:46.589498+0000","last_undegraded":"2024-04-26T21:09:46.589498+0000","last_fullsized":"2024-04-26T21:09:46.589498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:32.171086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104657+0000","last_change":"2024-04-26T21:09:41.079310+0000","last_active":"2024-04-26T21:09:42.104657+0000","last_peered":"2024-04-26T21:09:42.104657+0000","last_clean":"2024-04-26T21:09:42.104657+0000","last_became_active":"2024-04-26T21:09:41.077848+0000","last_became_peered":"2024-04-26T21:09:41.077848+0000","last_unstale":"2024-04-26T21:09:42.104657+0000","last_undegraded":"2024-04-26T21:09:42.104657+0000","last_fullsized":"2024-04-26T21:09:42.104657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:51.745349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.498276+0000","last_change":"2024-04-26T21:09:40.391216+0000","last_active":"2024-04-26T21:09:48.498276+0000","last_peered":"2024-04-26T21:09:48.498276+0000","last_clean":"2024-04-26T21:09:48.498276+0000","last_became_active":"2024-04-26T21:09:40.390753+0000","last_became_peered":"2024-04-26T21:09:40.390753+0000","last_unstale":"2024-04-26T21:09:48.498276+0000","last_undegraded":"2024-04-26T21:09:48.498276+0000","last_fullsized":"2024-04-26T21:09:48.498276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:24.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037992+0000","last_change":"2024-04-26T21:09:41.079881+0000","last_active":"2024-04-26T21:09:42.037992+0000","last_peered":"2024-04-26T21:09:42.037992+0000","last_clean":"2024-04-26T21:09:42.037992+0000","last_became_active":"2024-04-26T21:09:41.079506+0000","last_became_peered":"2024-04-26T21:09:41.079506+0000","last_unstale":"2024-04-26T21:09:42.037992+0000","last_undegraded":"2024-04-26T21:09:42.037992+0000","last_fullsized":"2024-04-26T21:09:42.037992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:56.300294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.491068+0000","last_change":"2024-04-26T21:09:40.391458+0000","last_active":"2024-04-26T21:09:50.491068+0000","last_peered":"2024-04-26T21:09:50.491068+0000","last_clean":"2024-04-26T21:09:50.491068+0000","last_became_active":"2024-04-26T21:09:40.388494+0000","last_became_peered":"2024-04-26T21:09:40.388494+0000","last_unstale":"2024-04-26T21:09:50.491068+0000","last_undegraded":"2024-04-26T21:09:50.491068+0000","last_fullsized":"2024-04-26T21:09:50.491068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:07.523885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036841+0000","last_change":"2024-04-26T21:09:41.068278+0000","last_active":"2024-04-26T21:09:42.036841+0000","last_peered":"2024-04-26T21:09:42.036841+0000","last_clean":"2024-04-26T21:09:42.036841+0000","last_became_active":"2024-04-26T21:09:41.067865+0000","last_became_peered":"2024-04-26T21:09:41.067865+0000","last_unstale":"2024-04-26T21:09:42.036841+0000","last_undegraded":"2024-04-26T21:09:42.036841+0000","last_fullsized":"2024-04-26T21:09:42.036841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:19.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.476293+0000","last_change":"2024-04-26T21:09:40.394390+0000","last_active":"2024-04-26T21:09:48.476293+0000","last_peered":"2024-04-26T21:09:48.476293+0000","last_clean":"2024-04-26T21:09:48.476293+0000","last_became_active":"2024-04-26T21:09:40.389728+0000","last_became_peered":"2024-04-26T21:09:40.389728+0000","last_unstale":"2024-04-26T21:09:48.476293+0000","last_undegraded":"2024-04-26T21:09:48.476293+0000","last_fullsized":"2024-04-26T21:09:48.476293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:42.301974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105384+0000","last_change":"2024-04-26T21:09:41.075703+0000","last_active":"2024-04-26T21:09:42.105384+0000","last_peered":"2024-04-26T21:09:42.105384+0000","last_clean":"2024-04-26T21:09:42.105384+0000","last_became_active":"2024-04-26T21:09:41.075310+0000","last_became_peered":"2024-04-26T21:09:41.075310+0000","last_unstale":"2024-04-26T21:09:42.105384+0000","last_undegraded":"2024-04-26T21:09:42.105384+0000","last_fullsized":"2024-04-26T21:09:42.105384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:20.030941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.576222+0000","last_change":"2024-04-26T21:09:40.381293+0000","last_active":"2024-04-26T21:09:47.576222+0000","last_peered":"2024-04-26T21:09:47.576222+0000","last_clean":"2024-04-26T21:09:47.576222+0000","last_became_active":"2024-04-26T21:09:40.380287+0000","last_became_peered":"2024-04-26T21:09:40.380287+0000","last_unstale":"2024-04-26T21:09:47.576222+0000","last_undegraded":"2024-04-26T21:09:47.576222+0000","last_fullsized":"2024-04-26T21:09:47.576222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:47.479546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037097+0000","last_change":"2024-04-26T21:09:41.077928+0000","last_active":"2024-04-26T21:09:42.037097+0000","last_peered":"2024-04-26T21:09:42.037097+0000","last_clean":"2024-04-26T21:09:42.037097+0000","last_became_active":"2024-04-26T21:09:41.076611+0000","last_became_peered":"2024-04-26T21:09:41.076611+0000","last_unstale":"2024-04-26T21:09:42.037097+0000","last_undegraded":"2024-04-26T21:09:42.037097+0000","last_fullsized":"2024-04-26T21:09:42.037097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:43.893631+0000","last_change":"2024-04-26T21:09:40.394161+0000","last_active":"2024-04-26T21:09:43.893631+0000","last_peered":"2024-04-26T21:09:43.893631+0000","last_clean":"2024-04-26T21:09:43.893631+0000","last_became_active":"2024-04-26T21:09:40.393277+0000","last_became_peered":"2024-04-26T21:09:40.393277+0000","last_unstale":"2024-04-26T21:09:43.893631+0000","last_undegraded":"2024-04-26T21:09:43.893631+0000","last_fullsized":"2024-04-26T21:09:43.893631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:32.408080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105981+0000","last_change":"2024-04-26T21:09:41.070926+0000","last_active":"2024-04-26T21:09:42.105981+0000","last_peered":"2024-04-26T21:09:42.105981+0000","last_clean":"2024-04-26T21:09:42.105981+0000","last_became_active":"2024-04-26T21:09:41.070260+0000","last_became_peered":"2024-04-26T21:09:41.070260+0000","last_unstale":"2024-04-26T21:09:42.105981+0000","last_undegraded":"2024-04-26T21:09:42.105981+0000","last_fullsized":"2024-04-26T21:09:42.105981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:22.916960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.507585+0000","last_change":"2024-04-26T21:09:40.395088+0000","last_active":"2024-04-26T21:09:49.507585+0000","last_peered":"2024-04-26T21:09:49.507585+0000","last_clean":"2024-04-26T21:09:49.507585+0000","last_became_active":"2024-04-26T21:09:40.393921+0000","last_became_peered":"2024-04-26T21:09:40.393921+0000","last_unstale":"2024-04-26T21:09:49.507585+0000","last_undegraded":"2024-04-26T21:09:49.507585+0000","last_fullsized":"2024-04-26T21:09:49.507585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:30.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105610+0000","last_change":"2024-04-26T21:09:41.064834+0000","last_active":"2024-04-26T21:09:42.105610+0000","last_peered":"2024-04-26T21:09:42.105610+0000","last_clean":"2024-04-26T21:09:42.105610+0000","last_became_active":"2024-04-26T21:09:41.063875+0000","last_became_peered":"2024-04-26T21:09:41.063875+0000","last_unstale":"2024-04-26T21:09:42.105610+0000","last_undegraded":"2024-04-26T21:09:42.105610+0000","last_fullsized":"2024-04-26T21:09:42.105610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:48.438740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103236+0000","last_change":"2024-04-26T21:09:41.075181+0000","last_active":"2024-04-26T21:09:42.103236+0000","last_peered":"2024-04-26T21:09:42.103236+0000","last_clean":"2024-04-26T21:09:42.103236+0000","last_became_active":"2024-04-26T21:09:41.073137+0000","last_became_peered":"2024-04-26T21:09:41.073137+0000","last_unstale":"2024-04-26T21:09:42.103236+0000","last_undegraded":"2024-04-26T21:09:42.103236+0000","last_fullsized":"2024-04-26T21:09:42.103236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:08.609048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.589629+0000","last_change":"2024-04-26T21:09:40.404159+0000","last_active":"2024-04-26T21:09:46.589629+0000","last_peered":"2024-04-26T21:09:46.589629+0000","last_clean":"2024-04-26T21:09:46.589629+0000","last_became_active":"2024-04-26T21:09:40.403828+0000","last_became_peered":"2024-04-26T21:09:40.403828+0000","last_unstale":"2024-04-26T21:09:46.589629+0000","last_undegraded":"2024-04-26T21:09:46.589629+0000","last_fullsized":"2024-04-26T21:09:46.589629+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:51.245405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103873+0000","last_change":"2024-04-26T21:09:41.068913+0000","last_active":"2024-04-26T21:09:42.103873+0000","last_peered":"2024-04-26T21:09:42.103873+0000","last_clean":"2024-04-26T21:09:42.103873+0000","last_became_active":"2024-04-26T21:09:41.068534+0000","last_became_peered":"2024-04-26T21:09:41.068534+0000","last_unstale":"2024-04-26T21:09:42.103873+0000","last_undegraded":"2024-04-26T21:09:42.103873+0000","last_fullsized":"2024-04-26T21:09:42.103873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:39.739853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.498415+0000","last_change":"2024-04-26T21:09:40.387055+0000","last_active":"2024-04-26T21:09:48.498415+0000","last_peered":"2024-04-26T21:09:48.498415+0000","last_clean":"2024-04-26T21:09:48.498415+0000","last_became_active":"2024-04-26T21:09:40.386684+0000","last_became_peered":"2024-04-26T21:09:40.386684+0000","last_unstale":"2024-04-26T21:09:48.498415+0000","last_undegraded":"2024-04-26T21:09:48.498415+0000","last_fullsized":"2024-04-26T21:09:48.498415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:02.871494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036617+0000","last_change":"2024-04-26T21:08:05.429394+0000","last_active":"2024-04-26T21:09:42.036617+0000","last_peered":"2024-04-26T21:09:42.036617+0000","last_clean":"2024-04-26T21:09:42.036617+0000","last_became_active":"2024-04-26T21:08:05.428055+0000","last_became_peered":"2024-04-26T21:08:05.428055+0000","last_unstale":"2024-04-26T21:09:42.036617+0000","last_undegraded":"2024-04-26T21:09:42.036617+0000","last_fullsized":"2024-04-26T21:09:42.036617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_clean_scrub_stamp":"2024-04-26T21:03:22.595431+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:23.104153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104544+0000","last_change":"2024-04-26T21:09:41.080213+0000","last_active":"2024-04-26T21:09:42.104544+0000","last_peered":"2024-04-26T21:09:42.104544+0000","last_clean":"2024-04-26T21:09:42.104544+0000","last_became_active":"2024-04-26T21:09:41.076313+0000","last_became_peered":"2024-04-26T21:09:41.076313+0000","last_unstale":"2024-04-26T21:09:42.104544+0000","last_undegraded":"2024-04-26T21:09:42.104544+0000","last_fullsized":"2024-04-26T21:09:42.104544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:48.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.592191+0000","last_change":"2024-04-26T21:09:40.387563+0000","last_active":"2024-04-26T21:09:46.592191+0000","last_peered":"2024-04-26T21:09:46.592191+0000","last_clean":"2024-04-26T21:09:46.592191+0000","last_became_active":"2024-04-26T21:09:40.383243+0000","last_became_peered":"2024-04-26T21:09:40.383243+0000","last_unstale":"2024-04-26T21:09:46.592191+0000","last_undegraded":"2024-04-26T21:09:46.592191+0000","last_fullsized":"2024-04-26T21:09:46.592191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:16.905074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.476550+0000","last_change":"2024-04-26T21:09:40.398078+0000","last_active":"2024-04-26T21:09:48.476550+0000","last_peered":"2024-04-26T21:09:48.476550+0000","last_clean":"2024-04-26T21:09:48.476550+0000","last_became_active":"2024-04-26T21:09:40.397007+0000","last_became_peered":"2024-04-26T21:09:40.397007+0000","last_unstale":"2024-04-26T21:09:48.476550+0000","last_undegraded":"2024-04-26T21:09:48.476550+0000","last_fullsized":"2024-04-26T21:09:48.476550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:18.374678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105027+0000","last_change":"2024-04-26T21:09:41.063430+0000","last_active":"2024-04-26T21:09:42.105027+0000","last_peered":"2024-04-26T21:09:42.105027+0000","last_clean":"2024-04-26T21:09:42.105027+0000","last_became_active":"2024-04-26T21:09:41.062591+0000","last_became_peered":"2024-04-26T21:09:41.062591+0000","last_unstale":"2024-04-26T21:09:42.105027+0000","last_undegraded":"2024-04-26T21:09:42.105027+0000","last_fullsized":"2024-04-26T21:09:42.105027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:58.874576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.607299+0000","last_change":"2024-04-26T21:09:40.378465+0000","last_active":"2024-04-26T21:09:47.607299+0000","last_peered":"2024-04-26T21:09:47.607299+0000","last_clean":"2024-04-26T21:09:47.607299+0000","last_became_active":"2024-04-26T21:09:40.378089+0000","last_became_peered":"2024-04-26T21:09:40.378089+0000","last_unstale":"2024-04-26T21:09:47.607299+0000","last_undegraded":"2024-04-26T21:09:47.607299+0000","last_fullsized":"2024-04-26T21:09:47.607299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:56.425077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101610+0000","last_change":"2024-04-26T21:09:41.080446+0000","last_active":"2024-04-26T21:09:42.101610+0000","last_peered":"2024-04-26T21:09:42.101610+0000","last_clean":"2024-04-26T21:09:42.101610+0000","last_became_active":"2024-04-26T21:09:41.074363+0000","last_became_peered":"2024-04-26T21:09:41.074363+0000","last_unstale":"2024-04-26T21:09:42.101610+0000","last_undegraded":"2024-04-26T21:09:42.101610+0000","last_fullsized":"2024-04-26T21:09:42.101610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:14.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.497248+0000","last_change":"2024-04-26T21:09:40.403106+0000","last_active":"2024-04-26T21:09:48.497248+0000","last_peered":"2024-04-26T21:09:48.497248+0000","last_clean":"2024-04-26T21:09:48.497248+0000","last_became_active":"2024-04-26T21:09:40.402630+0000","last_became_peered":"2024-04-26T21:09:40.402630+0000","last_unstale":"2024-04-26T21:09:48.497248+0000","last_undegraded":"2024-04-26T21:09:48.497248+0000","last_fullsized":"2024-04-26T21:09:48.497248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:37.555164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102987+0000","last_change":"2024-04-26T21:09:41.069407+0000","last_active":"2024-04-26T21:09:42.102987+0000","last_peered":"2024-04-26T21:09:42.102987+0000","last_clean":"2024-04-26T21:09:42.102987+0000","last_became_active":"2024-04-26T21:09:41.067522+0000","last_became_peered":"2024-04-26T21:09:41.067522+0000","last_unstale":"2024-04-26T21:09:42.102987+0000","last_undegraded":"2024-04-26T21:09:42.102987+0000","last_fullsized":"2024-04-26T21:09:42.102987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.338665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.576252+0000","last_change":"2024-04-26T21:09:40.391323+0000","last_active":"2024-04-26T21:09:45.576252+0000","last_peered":"2024-04-26T21:09:45.576252+0000","last_clean":"2024-04-26T21:09:45.576252+0000","last_became_active":"2024-04-26T21:09:40.390982+0000","last_became_peered":"2024-04-26T21:09:40.390982+0000","last_unstale":"2024-04-26T21:09:45.576252+0000","last_undegraded":"2024-04-26T21:09:45.576252+0000","last_fullsized":"2024-04-26T21:09:45.576252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:03.465013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101799+0000","last_change":"2024-04-26T21:09:41.068305+0000","last_active":"2024-04-26T21:09:42.101799+0000","last_peered":"2024-04-26T21:09:42.101799+0000","last_clean":"2024-04-26T21:09:42.101799+0000","last_became_active":"2024-04-26T21:09:41.067946+0000","last_became_peered":"2024-04-26T21:09:41.067946+0000","last_unstale":"2024-04-26T21:09:42.101799+0000","last_undegraded":"2024-04-26T21:09:42.101799+0000","last_fullsized":"2024-04-26T21:09:42.101799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:38.761142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.497595+0000","last_change":"2024-04-26T21:09:40.387004+0000","last_active":"2024-04-26T21:09:48.497595+0000","last_peered":"2024-04-26T21:09:48.497595+0000","last_clean":"2024-04-26T21:09:48.497595+0000","last_became_active":"2024-04-26T21:09:40.382385+0000","last_became_peered":"2024-04-26T21:09:40.382385+0000","last_unstale":"2024-04-26T21:09:48.497595+0000","last_undegraded":"2024-04-26T21:09:48.497595+0000","last_fullsized":"2024-04-26T21:09:48.497595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:40.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036681+0000","last_change":"2024-04-26T21:09:41.071024+0000","last_active":"2024-04-26T21:09:42.036681+0000","last_peered":"2024-04-26T21:09:42.036681+0000","last_clean":"2024-04-26T21:09:42.036681+0000","last_became_active":"2024-04-26T21:09:41.070456+0000","last_became_peered":"2024-04-26T21:09:41.070456+0000","last_unstale":"2024-04-26T21:09:42.036681+0000","last_undegraded":"2024-04-26T21:09:42.036681+0000","last_fullsized":"2024-04-26T21:09:42.036681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:56.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:46.563470+0000","last_change":"2024-04-26T21:09:40.404128+0000","last_active":"2024-04-26T21:09:46.563470+0000","last_peered":"2024-04-26T21:09:46.563470+0000","last_clean":"2024-04-26T21:09:46.563470+0000","last_became_active":"2024-04-26T21:09:40.403760+0000","last_became_peered":"2024-04-26T21:09:40.403760+0000","last_unstale":"2024-04-26T21:09:46.563470+0000","last_undegraded":"2024-04-26T21:09:46.563470+0000","last_fullsized":"2024-04-26T21:09:46.563470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103396+0000","last_change":"2024-04-26T21:09:41.067676+0000","last_active":"2024-04-26T21:09:42.103396+0000","last_peered":"2024-04-26T21:09:42.103396+0000","last_clean":"2024-04-26T21:09:42.103396+0000","last_became_active":"2024-04-26T21:09:41.066736+0000","last_became_peered":"2024-04-26T21:09:41.066736+0000","last_unstale":"2024-04-26T21:09:42.103396+0000","last_undegraded":"2024-04-26T21:09:42.103396+0000","last_fullsized":"2024-04-26T21:09:42.103396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:39.477690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:45.577682+0000","last_change":"2024-04-26T21:09:40.392845+0000","last_active":"2024-04-26T21:09:45.577682+0000","last_peered":"2024-04-26T21:09:45.577682+0000","last_clean":"2024-04-26T21:09:45.577682+0000","last_became_active":"2024-04-26T21:09:40.392488+0000","last_became_peered":"2024-04-26T21:09:40.392488+0000","last_unstale":"2024-04-26T21:09:45.577682+0000","last_undegraded":"2024-04-26T21:09:45.577682+0000","last_fullsized":"2024-04-26T21:09:45.577682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:54.493294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037601+0000","last_change":"2024-04-26T21:09:41.074270+0000","last_active":"2024-04-26T21:09:42.037601+0000","last_peered":"2024-04-26T21:09:42.037601+0000","last_clean":"2024-04-26T21:09:42.037601+0000","last_became_active":"2024-04-26T21:09:41.073847+0000","last_became_peered":"2024-04-26T21:09:41.073847+0000","last_unstale":"2024-04-26T21:09:42.037601+0000","last_undegraded":"2024-04-26T21:09:42.037601+0000","last_fullsized":"2024-04-26T21:09:42.037601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:03.737883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:47.606891+0000","last_change":"2024-04-26T21:09:40.397644+0000","last_active":"2024-04-26T21:09:47.606891+0000","last_peered":"2024-04-26T21:09:47.606891+0000","last_clean":"2024-04-26T21:09:47.606891+0000","last_became_active":"2024-04-26T21:09:40.397234+0000","last_became_peered":"2024-04-26T21:09:40.397234+0000","last_unstale":"2024-04-26T21:09:47.606891+0000","last_undegraded":"2024-04-26T21:09:47.606891+0000","last_fullsized":"2024-04-26T21:09:47.606891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:32.220330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038690+0000","last_change":"2024-04-26T21:09:41.072171+0000","last_active":"2024-04-26T21:09:42.038690+0000","last_peered":"2024-04-26T21:09:42.038690+0000","last_clean":"2024-04-26T21:09:42.038690+0000","last_became_active":"2024-04-26T21:09:41.071706+0000","last_became_peered":"2024-04-26T21:09:41.071706+0000","last_unstale":"2024-04-26T21:09:42.038690+0000","last_undegraded":"2024-04-26T21:09:42.038690+0000","last_fullsized":"2024-04-26T21:09:42.038690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:14.286865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:48.476689+0000","last_change":"2024-04-26T21:09:40.393724+0000","last_active":"2024-04-26T21:09:48.476689+0000","last_peered":"2024-04-26T21:09:48.476689+0000","last_clean":"2024-04-26T21:09:48.476689+0000","last_became_active":"2024-04-26T21:09:40.392441+0000","last_became_peered":"2024-04-26T21:09:40.392441+0000","last_unstale":"2024-04-26T21:09:48.476689+0000","last_undegraded":"2024-04-26T21:09:48.476689+0000","last_fullsized":"2024-04-26T21:09:48.476689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:11.989418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.037807+0000","last_change":"2024-04-26T21:09:41.071314+0000","last_active":"2024-04-26T21:09:42.037807+0000","last_peered":"2024-04-26T21:09:42.037807+0000","last_clean":"2024-04-26T21:09:42.037807+0000","last_became_active":"2024-04-26T21:09:41.070874+0000","last_became_peered":"2024-04-26T21:09:41.070874+0000","last_unstale":"2024-04-26T21:09:42.037807+0000","last_undegraded":"2024-04-26T21:09:42.037807+0000","last_fullsized":"2024-04-26T21:09:42.037807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:33.877338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.792361+0000","last_change":"2024-04-26T21:09:40.400884+0000","last_active":"2024-04-26T21:09:42.792361+0000","last_peered":"2024-04-26T21:09:42.792361+0000","last_clean":"2024-04-26T21:09:42.792361+0000","last_became_active":"2024-04-26T21:09:40.400326+0000","last_became_peered":"2024-04-26T21:09:40.400326+0000","last_unstale":"2024-04-26T21:09:42.792361+0000","last_undegraded":"2024-04-26T21:09:42.792361+0000","last_fullsized":"2024-04-26T21:09:42.792361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:13.114233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.038348+0000","last_change":"2024-04-26T21:09:41.071959+0000","last_active":"2024-04-26T21:09:42.038348+0000","last_peered":"2024-04-26T21:09:42.038348+0000","last_clean":"2024-04-26T21:09:42.038348+0000","last_became_active":"2024-04-26T21:09:41.071601+0000","last_became_peered":"2024-04-26T21:09:41.071601+0000","last_unstale":"2024-04-26T21:09:42.038348+0000","last_undegraded":"2024-04-26T21:09:42.038348+0000","last_fullsized":"2024-04-26T21:09:42.038348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:29.806956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036534+0000","last_change":"2024-04-26T21:09:41.066764+0000","last_active":"2024-04-26T21:09:42.036534+0000","last_peered":"2024-04-26T21:09:42.036534+0000","last_clean":"2024-04-26T21:09:42.036534+0000","last_became_active":"2024-04-26T21:09:41.066333+0000","last_became_peered":"2024-04-26T21:09:41.066333+0000","last_unstale":"2024-04-26T21:09:42.036534+0000","last_undegraded":"2024-04-26T21:09:42.036534+0000","last_fullsized":"2024-04-26T21:09:42.036534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:32.680215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.099875+0000","last_change":"2024-04-26T21:09:41.072567+0000","last_active":"2024-04-26T21:09:42.099875+0000","last_peered":"2024-04-26T21:09:42.099875+0000","last_clean":"2024-04-26T21:09:42.099875+0000","last_became_active":"2024-04-26T21:09:41.072033+0000","last_became_peered":"2024-04-26T21:09:41.072033+0000","last_unstale":"2024-04-26T21:09:42.099875+0000","last_undegraded":"2024-04-26T21:09:42.099875+0000","last_fullsized":"2024-04-26T21:09:42.099875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:37.178970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100637+0000","last_change":"2024-04-26T21:09:41.071111+0000","last_active":"2024-04-26T21:09:42.100637+0000","last_peered":"2024-04-26T21:09:42.100637+0000","last_clean":"2024-04-26T21:09:42.100637+0000","last_became_active":"2024-04-26T21:09:41.070354+0000","last_became_peered":"2024-04-26T21:09:41.070354+0000","last_unstale":"2024-04-26T21:09:42.100637+0000","last_undegraded":"2024-04-26T21:09:42.100637+0000","last_fullsized":"2024-04-26T21:09:42.100637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:10.990058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101560+0000","last_change":"2024-04-26T21:09:41.076927+0000","last_active":"2024-04-26T21:09:42.101560+0000","last_peered":"2024-04-26T21:09:42.101560+0000","last_clean":"2024-04-26T21:09:42.101560+0000","last_became_active":"2024-04-26T21:09:41.076600+0000","last_became_peered":"2024-04-26T21:09:41.076600+0000","last_unstale":"2024-04-26T21:09:42.101560+0000","last_undegraded":"2024-04-26T21:09:42.101560+0000","last_fullsized":"2024-04-26T21:09:42.101560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:25.960409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.104838+0000","last_change":"2024-04-26T21:09:41.079293+0000","last_active":"2024-04-26T21:09:42.104838+0000","last_peered":"2024-04-26T21:09:42.104838+0000","last_clean":"2024-04-26T21:09:42.104838+0000","last_became_active":"2024-04-26T21:09:41.077473+0000","last_became_peered":"2024-04-26T21:09:41.077473+0000","last_unstale":"2024-04-26T21:09:42.104838+0000","last_undegraded":"2024-04-26T21:09:42.104838+0000","last_fullsized":"2024-04-26T21:09:42.104838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:47.281535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036886+0000","last_change":"2024-04-26T21:09:41.065914+0000","last_active":"2024-04-26T21:09:42.036886+0000","last_peered":"2024-04-26T21:09:42.036886+0000","last_clean":"2024-04-26T21:09:42.036886+0000","last_became_active":"2024-04-26T21:09:41.065477+0000","last_became_peered":"2024-04-26T21:09:41.065477+0000","last_unstale":"2024-04-26T21:09:42.036886+0000","last_undegraded":"2024-04-26T21:09:42.036886+0000","last_fullsized":"2024-04-26T21:09:42.036886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:40.517174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.103823+0000","last_change":"2024-04-26T21:09:41.068594+0000","last_active":"2024-04-26T21:09:42.103823+0000","last_peered":"2024-04-26T21:09:42.103823+0000","last_clean":"2024-04-26T21:09:42.103823+0000","last_became_active":"2024-04-26T21:09:41.068238+0000","last_became_peered":"2024-04-26T21:09:41.068238+0000","last_unstale":"2024-04-26T21:09:42.103823+0000","last_undegraded":"2024-04-26T21:09:42.103823+0000","last_fullsized":"2024-04-26T21:09:42.103823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:58.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.101141+0000","last_change":"2024-04-26T21:09:41.081684+0000","last_active":"2024-04-26T21:09:42.101141+0000","last_peered":"2024-04-26T21:09:42.101141+0000","last_clean":"2024-04-26T21:09:42.101141+0000","last_became_active":"2024-04-26T21:09:41.074963+0000","last_became_peered":"2024-04-26T21:09:41.074963+0000","last_unstale":"2024-04-26T21:09:42.101141+0000","last_undegraded":"2024-04-26T21:09:42.101141+0000","last_fullsized":"2024-04-26T21:09:42.101141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:14.736390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102778+0000","last_change":"2024-04-26T21:09:41.068052+0000","last_active":"2024-04-26T21:09:42.102778+0000","last_peered":"2024-04-26T21:09:42.102778+0000","last_clean":"2024-04-26T21:09:42.102778+0000","last_became_active":"2024-04-26T21:09:41.067728+0000","last_became_peered":"2024-04-26T21:09:41.067728+0000","last_unstale":"2024-04-26T21:09:42.102778+0000","last_undegraded":"2024-04-26T21:09:42.102778+0000","last_fullsized":"2024-04-26T21:09:42.102778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:19.186963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.100313+0000","last_change":"2024-04-26T21:09:41.078725+0000","last_active":"2024-04-26T21:09:42.100313+0000","last_peered":"2024-04-26T21:09:42.100313+0000","last_clean":"2024-04-26T21:09:42.100313+0000","last_became_active":"2024-04-26T21:09:41.075757+0000","last_became_peered":"2024-04-26T21:09:41.075757+0000","last_unstale":"2024-04-26T21:09:42.100313+0000","last_undegraded":"2024-04-26T21:09:42.100313+0000","last_fullsized":"2024-04-26T21:09:42.100313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:50.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.102664+0000","last_change":"2024-04-26T21:09:41.067872+0000","last_active":"2024-04-26T21:09:42.102664+0000","last_peered":"2024-04-26T21:09:42.102664+0000","last_clean":"2024-04-26T21:09:42.102664+0000","last_became_active":"2024-04-26T21:09:41.067476+0000","last_became_peered":"2024-04-26T21:09:41.067476+0000","last_unstale":"2024-04-26T21:09:42.102664+0000","last_undegraded":"2024-04-26T21:09:42.102664+0000","last_fullsized":"2024-04-26T21:09:42.102664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:54.740719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.105327+0000","last_change":"2024-04-26T21:09:41.080571+0000","last_active":"2024-04-26T21:09:42.105327+0000","last_peered":"2024-04-26T21:09:42.105327+0000","last_clean":"2024-04-26T21:09:42.105327+0000","last_became_active":"2024-04-26T21:09:41.078465+0000","last_became_peered":"2024-04-26T21:09:41.078465+0000","last_unstale":"2024-04-26T21:09:42.105327+0000","last_undegraded":"2024-04-26T21:09:42.105327+0000","last_fullsized":"2024-04-26T21:09:42.105327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:27.636693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:09:42.036697+0000","last_change":"2024-04-26T21:09:41.080111+0000","last_active":"2024-04-26T21:09:42.036697+0000","last_peered":"2024-04-26T21:09:42.036697+0000","last_clean":"2024-04-26T21:09:42.036697+0000","last_became_active":"2024-04-26T21:09:41.079220+0000","last_became_peered":"2024-04-26T21:09:41.079220+0000","last_unstale":"2024-04-26T21:09:42.036697+0000","last_undegraded":"2024-04-26T21:09:42.036697+0000","last_fullsized":"2024-04-26T21:09:42.036697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.055957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":51018,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":49,"num_read_kb":75,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":62,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":85841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":453017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547224,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1084675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710750,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":453343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972391,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":453935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266733,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1046453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":452525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":475082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":455397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":452843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888131,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":453526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691900,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1585152,"data_stored":1045623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":46354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:09:52.304 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:09:52.304 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T21:09:52.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T21:09:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:52.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:52 smithi027 ceph-mon[28085]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:53.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:52 smithi204 ceph-mon[32145]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:53.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:52 smithi094 ceph-mon[31273]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:53.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:53 smithi027 ceph-mon[28085]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-26T21:09:53.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:53 smithi027 ceph-mon[28085]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:53.927 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:53 smithi094 ceph-mon[31273]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-26T21:09:53.927 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:53 smithi094 ceph-mon[31273]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:54.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:53 smithi204 ceph-mon[32145]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-26T21:09:54.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:53 smithi204 ceph-mon[32145]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:09:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 -- 172.21.15.27:0/412178340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 msgr2=0x7fd9a0105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/412178340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a0105920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd99000b0d0 tx=0x7fd99000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 -- 172.21.15.27:0/412178340 shutdown_connections 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/412178340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a01070a0 0x7fd9a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/412178340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a0105920 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/412178340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a0106700 0x7fd9a0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 -- 172.21.15.27:0/412178340 >> 172.21.15.27:0/412178340 conn(0x7fd9a0100aa0 msgr2=0x7fd9a0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 -- 172.21.15.27:0/412178340 shutdown_connections 2024-04-26T21:09:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.255+0000 7fd9a5c00700 1 -- 172.21.15.27:0/412178340 wait complete. 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.256+0000 7fd9a5c00700 1 Processor -- start 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.256+0000 7fd9a5c00700 1 -- start start 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a0106700 0x7fd9a019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a01070a0 0x7fd9a01a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0110e50 con 0x7fd9a0105500 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0110cd0 con 0x7fd9a0106700 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0110fd0 con 0x7fd9a01070a0 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd99fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd99fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36378/0 (socket says 172.21.15.27:36378) 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd99fe00700 1 -- 172.21.15.27:0/3477822587 learned_addr learned my addr 172.21.15.27:0/3477822587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd997400700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a0106700 0x7fd9a019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.257+0000 7fd9a5200700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a01070a0 0x7fd9a01a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 -- 172.21.15.27:0/3477822587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a01070a0 msgr2=0x7fd9a01a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a01070a0 0x7fd9a01a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 -- 172.21.15.27:0/3477822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a0106700 msgr2=0x7fd9a019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a0106700 0x7fd9a019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 -- 172.21.15.27:0/3477822587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a01a30d0 con 0x7fd9a0105500 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99fe00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd990009830 tx=0x7fd990009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99001f030 con 0x7fd9a0105500 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd990007b40 con 0x7fd9a0105500 2024-04-26T21:09:55.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a01a3360 con 0x7fd9a0105500 2024-04-26T21:09:55.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.258+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99001b690 con 0x7fd9a0105500 2024-04-26T21:09:55.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.259+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a01a3750 con 0x7fd9a0105500 2024-04-26T21:09:55.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.260+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a0064a80 con 0x7fd9a0105500 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.262+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd99001b830 con 0x7fd9a0105500 2024-04-26T21:09:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.264+0000 7fd99d600700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 0x7fd9840a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.264+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd99000f020 con 0x7fd9a0105500 2024-04-26T21:09:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.264+0000 7fd997400700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 0x7fd9840a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.265+0000 7fd997400700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 0x7fd9840a6240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd988005950 tx=0x7fd9880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.269+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd990029020 con 0x7fd9a0105500 2024-04-26T21:09:55.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.595+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd9a002e530 con 0x7fd9a0105500 2024-04-26T21:09:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.601+0000 7fd99d600700 1 -- 172.21.15.27:0/3477822587 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fd9a002e530 con 0x7fd9a0105500 2024-04-26T21:09:55.602 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-04-26T21:09:55.602 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAjGCxm8jCcIxAAAxpzEmlN3ndsSuGL9+6EKw== 2024-04-26T21:09:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 msgr2=0x7fd9840a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 0x7fd9840a6240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd988005950 tx=0x7fd9880058e0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 msgr2=0x7fd9a019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd990009830 tx=0x7fd990009c30 comp rx=0 tx=0).stop 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 shutdown_connections 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd9840a3d80 0x7fd9840a6240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9a01070a0 0x7fd9a01a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9a0105500 0x7fd9a019b470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 --2- 172.21.15.27:0/3477822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9a0106700 0x7fd9a019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 >> 172.21.15.27:0/3477822587 conn(0x7fd9a0100aa0 msgr2=0x7fd9a0109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 shutdown_connections 2024-04-26T21:09:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:55.604+0000 7fd9a5c00700 1 -- 172.21.15.27:0/3477822587 wait complete. 2024-04-26T21:09:55.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:55 smithi027 ceph-mon[28085]: pgmap v374: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.4 KiB/s wr, 24 op/s 2024-04-26T21:09:55.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:55 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:09:55.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:55 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:09:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:55 smithi204 ceph-mon[32145]: pgmap v374: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.4 KiB/s wr, 24 op/s 2024-04-26T21:09:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:55 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:09:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:55 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:09:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:55 smithi094 ceph-mon[31273]: pgmap v374: 97 pgs: 97 active+clean; 627 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.4 KiB/s wr, 24 op/s 2024-04-26T21:09:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:55 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:09:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:55 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3477822587' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:09:56.428 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:09:56.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T21:09:56.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:09:56.569 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:09:56.569 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:09:56.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T21:09:57.092 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:09:57.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:57 smithi027 ceph-mon[28085]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-04-26T21:09:58.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:57 smithi204 ceph-mon[32145]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-04-26T21:09:58.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:57 smithi094 ceph-mon[31273]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-04-26T21:09:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 -- 172.21.15.27:0/4156868164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075f70 msgr2=0x7f96981140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 --2- 172.21.15.27:0/4156868164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075f70 0x7f96981140b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684009ca0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 -- 172.21.15.27:0/4156868164 shutdown_connections 2024-04-26T21:09:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 --2- 172.21.15.27:0/4156868164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96981145f0 0x7f96981169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 --2- 172.21.15.27:0/4156868164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9698075650 0x7f9698075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 --2- 172.21.15.27:0/4156868164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075f70 0x7f96981140b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 -- 172.21.15.27:0/4156868164 >> 172.21.15.27:0/4156868164 conn(0x7f969807a580 msgr2=0x7f969807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 -- 172.21.15.27:0/4156868164 shutdown_connections 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.314+0000 7f96a0600700 1 -- 172.21.15.27:0/4156868164 wait complete. 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.315+0000 7f96a0600700 1 Processor -- start 2024-04-26T21:09:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.315+0000 7f96a0600700 1 -- start start 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698075f70 0x7f96981a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96981145f0 0x7f96981a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9698119ab0 con 0x7f9698075650 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9698119930 con 0x7f96981145f0 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f96a0600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9698119c30 con 0x7f9698075f70 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f969dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f969dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54446/0 (socket says 172.21.15.27:54446) 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f969dc00700 1 -- 172.21.15.27:0/2249309579 learned_addr learned my addr 172.21.15.27:0/2249309579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.316+0000 7f969d200700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698075f70 0x7f96981a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969e600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96981145f0 0x7f96981a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 -- 172.21.15.27:0/2249309579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698075f70 msgr2=0x7f96981a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698075f70 0x7f96981a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 -- 172.21.15.27:0/2249309579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96981145f0 msgr2=0x7f96981a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96981145f0 0x7f96981a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 -- 172.21.15.27:0/2249309579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981a4ff0 con 0x7f9698075650 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f969dc00700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c9b0 tx=0x7f968c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c007b00 con 0x7f9698075650 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f968c007ca0 con 0x7f9698075650 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981a52e0 con 0x7f9698075650 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.317+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c018660 con 0x7f9698075650 2024-04-26T21:09:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.318+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981acb60 con 0x7f9698075650 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.319+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969806a110 con 0x7f9698075650 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.321+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f968c00f450 con 0x7f9698075650 2024-04-26T21:09:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.322+0000 7f9696000700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 0x7f96800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.323+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f968c0d0510 con 0x7f9698075650 2024-04-26T21:09:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.323+0000 7f969d200700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 0x7f96800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.323+0000 7f969d200700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 0x7f96800a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9684005850 tx=0x7f968401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.327+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f968c014030 con 0x7f9698075650 2024-04-26T21:09:59.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.611+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f96981acdf0 con 0x7f9698075650 2024-04-26T21:09:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.629+0000 7f9696000700 1 -- 172.21.15.27:0/2249309579 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7f968c09ce40 con 0x7f9698075650 2024-04-26T21:09:59.634 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:09:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.635+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 msgr2=0x7f96800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.635+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 0x7f96800a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9684005850 tx=0x7f968401a040 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 msgr2=0x7f969819ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c9b0 tx=0x7f968c00cd70 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 shutdown_connections 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f96800a3d50 0x7f96800a6210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96981145f0 0x7f96981a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9698075650 0x7f969819ffe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 --2- 172.21.15.27:0/2249309579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698075f70 0x7f96981a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 >> 172.21.15.27:0/2249309579 conn(0x7f969807a580 msgr2=0x7f9698113420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 shutdown_connections 2024-04-26T21:09:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:09:59.636+0000 7f96a0600700 1 -- 172.21.15.27:0/2249309579 wait complete. 2024-04-26T21:09:59.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:59 smithi027 ceph-mon[28085]: pgmap v376: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-04-26T21:09:59.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:09:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2249309579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:10:00.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:59 smithi204 ceph-mon[32145]: pgmap v376: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-04-26T21:10:00.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:09:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2249309579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:10:00.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:59 smithi094 ceph-mon[31273]: pgmap v376: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-04-26T21:10:00.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:09:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2249309579' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:10:00.735 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3365757774},{"type":"v1","addr":"172.21.15.27:6801","nonce":3365757774}]},"active_addr":"172.21.15.27:6801/3365757774","active_change":"2024-04-26T21:00:01.870881+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2356002952}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":238118775}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":542903082}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1196187505}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":137865064}]}]} 2024-04-26T21:10:00.740 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T21:10:00.741 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T21:10:00.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:10:00.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:00 smithi027 ceph-mon[28085]: overall HEALTH_OK 2024-04-26T21:10:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:00 smithi204 ceph-mon[32145]: overall HEALTH_OK 2024-04-26T21:10:01.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:00 smithi094 ceph-mon[31273]: overall HEALTH_OK 2024-04-26T21:10:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:01.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:01 smithi027 ceph-mon[28085]: pgmap v377: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-04-26T21:10:02.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:01 smithi204 ceph-mon[32145]: pgmap v377: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-04-26T21:10:02.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:01 smithi094 ceph-mon[31273]: pgmap v377: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T21:10:02.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:02 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:10:03.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:10:03.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:10:03.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T21:10:03.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:02 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-26T21:10:03.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:02 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:10:03.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 -- 172.21.15.27:0/1138365740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec41141d0 msgr2=0x7feec41165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 --2- 172.21.15.27:0/1138365740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec41141d0 0x7feec41165c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b0a0 tx=0x7feebc00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 -- 172.21.15.27:0/1138365740 shutdown_connections 2024-04-26T21:10:03.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 --2- 172.21.15.27:0/1138365740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec41141d0 0x7feec41165c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 --2- 172.21.15.27:0/1138365740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4077950 0x7feec4077d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 --2- 172.21.15.27:0/1138365740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec4078270 0x7feec4113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 -- 172.21.15.27:0/1138365740 >> 172.21.15.27:0/1138365740 conn(0x7feec4101060 msgr2=0x7feec4103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.417+0000 7feec9e00700 1 -- 172.21.15.27:0/1138365740 shutdown_connections 2024-04-26T21:10:03.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.418+0000 7feec9e00700 1 -- 172.21.15.27:0/1138365740 wait complete. 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 Processor -- start 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 -- start start 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 0x7feec41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 0x7feec41a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feec4119a90 con 0x7feec4077950 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feec4119910 con 0x7feec4078270 2024-04-26T21:10:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.419+0000 7feec9e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feec4119c10 con 0x7feec41141d0 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54466/0 (socket says 172.21.15.27:54466) 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 -- 172.21.15.27:0/2132198088 learned_addr learned my addr 172.21.15.27:0/2132198088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec3e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 0x7feec41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec9400700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 0x7feec41a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 -- 172.21.15.27:0/2132198088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 msgr2=0x7feec41a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 0x7feec41a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 -- 172.21.15.27:0/2132198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 msgr2=0x7feec41a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 0x7feec41a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec8a00700 1 -- 172.21.15.27:0/2132198088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec41a4f10 con 0x7feec4077950 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec3e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 0x7feec41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.420+0000 7feec9400700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 0x7feec41a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec8a00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c9b0 tx=0x7feeb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8007b00 con 0x7feec4077950 2024-04-26T21:10:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feeb8007ca0 con 0x7feec4077950 2024-04-26T21:10:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8018660 con 0x7feec4077950 2024-04-26T21:10:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec41a5200 con 0x7feec4077950 2024-04-26T21:10:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.421+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41aca80 con 0x7feec4077950 2024-04-26T21:10:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.422+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec406a110 con 0x7feec4077950 2024-04-26T21:10:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.424+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feeb800f450 con 0x7feec4077950 2024-04-26T21:10:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.426+0000 7feec1600700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 0x7feeac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.426+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7feeb80d09a0 con 0x7feec4077950 2024-04-26T21:10:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.426+0000 7feec3e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 0x7feeac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.427+0000 7feec3e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 0x7feeac0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feec407a8d0 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.429+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feeb8014030 con 0x7feec4077950 2024-04-26T21:10:03.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.664+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feec41a1260 con 0x7feec4077950 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.667+0000 7feec1600700 1 -- 172.21.15.27:0/2132198088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22670 (secure 0 0 0) 0x7feeb809d240 con 0x7feec4077950 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:10:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":89,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:02.655755+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 msgr2=0x7feeac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 0x7feeac0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feec407a8d0 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2024-04-26T21:10:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 msgr2=0x7feec419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c9b0 tx=0x7feeb800cd70 comp rx=0 tx=0).stop 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 shutdown_connections 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feeac0a3cd0 0x7feeac0a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feec41141d0 0x7feec41a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feec4077950 0x7feec419ff00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 --2- 172.21.15.27:0/2132198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec4078270 0x7feec41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.670+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 >> 172.21.15.27:0/2132198088 conn(0x7feec4101060 msgr2=0x7feec4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.671+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 shutdown_connections 2024-04-26T21:10:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:03.671+0000 7feec9e00700 1 -- 172.21.15.27:0/2132198088 wait complete. 2024-04-26T21:10:03.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: pgmap v378: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-26T21:10:03.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:10:03.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:03 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2132198088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: pgmap v378: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-26T21:10:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T21:10:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:10:04.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:03 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2132198088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: pgmap v378: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-26T21:10:04.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T21:10:04.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:10:04.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:03 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2132198088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:04.767 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T21:10:04.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:10:04.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:04 smithi027 ceph-mon[28085]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:10:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:04 smithi204 ceph-mon[32145]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:10:05.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:04 smithi094 ceph-mon[31273]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:10:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:05.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:05 smithi027 ceph-mon[28085]: pgmap v381: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:05.991 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:05 smithi094 ceph-mon[31273]: pgmap v381: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:05 smithi204 ceph-mon[32145]: pgmap v381: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 -- 172.21.15.27:0/696438568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 msgr2=0x7f8b5010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 --2- 172.21.15.27:0/696438568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5010ba20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009990 tx=0x7f8b40009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 -- 172.21.15.27:0/696438568 shutdown_connections 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 --2- 172.21.15.27:0/696438568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 0x7f8b50112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 --2- 172.21.15.27:0/696438568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5010ba20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 --2- 172.21.15.27:0/696438568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 0x7f8b50105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 -- 172.21.15.27:0/696438568 >> 172.21.15.27:0/696438568 conn(0x7f8b50101130 msgr2=0x7f8b50103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 -- 172.21.15.27:0/696438568 shutdown_connections 2024-04-26T21:10:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.360+0000 7f8b55a00700 1 -- 172.21.15.27:0/696438568 wait complete. 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.361+0000 7f8b55a00700 1 Processor -- start 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 -- start start 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 0x7f8b501a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 0x7f8b501a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50115740 con 0x7f8b5010b640 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501155c0 con 0x7f8b50105640 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b55a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501158c0 con 0x7f8b50106000 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b4f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b4f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54484/0 (socket says 172.21.15.27:54484) 2024-04-26T21:10:07.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.362+0000 7f8b4f400700 1 -- 172.21.15.27:0/3659406066 learned_addr learned my addr 172.21.15.27:0/3659406066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4ea00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 0x7f8b501a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4e000700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 0x7f8b501a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 -- 172.21.15.27:0/3659406066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 msgr2=0x7f8b501a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 0x7f8b501a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 -- 172.21.15.27:0/3659406066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 msgr2=0x7f8b501a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 0x7f8b501a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 -- 172.21.15.27:0/3659406066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5019c1c0 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.363+0000 7f8b4f400700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007ee0 tx=0x7f8b4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.364+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800dbd0 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.364+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5019c4b0 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.364+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b4800dd70 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.364+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b480058d0 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.364+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b501a8770 con 0x7f8b5010b640 2024-04-26T21:10:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.365+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5006a110 con 0x7f8b5010b640 2024-04-26T21:10:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.367+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b4801a020 con 0x7f8b5010b640 2024-04-26T21:10:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.368+0000 7f8b47400700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 0x7f8b340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.369+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f8b480cfc70 con 0x7f8b5010b640 2024-04-26T21:10:07.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.369+0000 7f8b4ea00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 0x7f8b340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.370+0000 7f8b4ea00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 0x7f8b340a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b40003760 tx=0x7f8b4000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.373+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b48010070 con 0x7f8b5010b640 2024-04-26T21:10:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.609+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b5002e490 con 0x7f8b5010b640 2024-04-26T21:10:07.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.611+0000 7f8b47400700 1 -- 172.21.15.27:0/3659406066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7f8b4809c480 con 0x7f8b5010b640 2024-04-26T21:10:07.612 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:10:07.612 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.613+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 msgr2=0x7f8b340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.613+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 0x7f8b340a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b40003760 tx=0x7f8b4000b540 comp rx=0 tx=0).stop 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 msgr2=0x7f8b5019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007ee0 tx=0x7f8b4800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 shutdown_connections 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8b340a3d50 0x7f8b340a6210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b5010b640 0x7f8b5019ba60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b50105640 0x7f8b501a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 --2- 172.21.15.27:0/3659406066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b50106000 0x7f8b501a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 >> 172.21.15.27:0/3659406066 conn(0x7f8b50101130 msgr2=0x7f8b50101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 shutdown_connections 2024-04-26T21:10:07.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:07.614+0000 7f8b55a00700 1 -- 172.21.15.27:0/3659406066 wait complete. 2024-04-26T21:10:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:07 smithi027 ceph-mon[28085]: pgmap v382: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:07 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3659406066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:07 smithi204 ceph-mon[32145]: pgmap v382: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:07 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3659406066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:08.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:07 smithi094 ceph-mon[31273]: pgmap v382: 97 pgs: 97 active+clean; 630 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s; 83 B/s, 0 objects/s recovering 2024-04-26T21:10:08.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:07 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3659406066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:10:08.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T21:10:08.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T21:10:08.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T21:10:08.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T21:10:08.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T21:10:08.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T21:10:08.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T21:10:08.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T21:10:08.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T21:10:08.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T21:10:08.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T21:10:08.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T21:10:09.604 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:09.609 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:09.635 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:09.718 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:09 smithi027 ceph-mon[28085]: pgmap v383: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 9 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:09 smithi204 ceph-mon[32145]: pgmap v383: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 9 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:09 smithi094 ceph-mon[31273]: pgmap v383: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 9 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:11.267 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:11.269 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:11.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:11 smithi027 ceph-mon[28085]: pgmap v384: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 639 B/s wr, 5 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:11 smithi204 ceph-mon[32145]: pgmap v384: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 639 B/s wr, 5 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:12.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:11 smithi094 ceph-mon[31273]: pgmap v384: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 639 B/s wr, 5 op/s; 6.0 KiB/s, 2 objects/s recovering 2024-04-26T21:10:13.899 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:13 smithi027 ceph-mon[28085]: pgmap v385: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 545 B/s wr, 7 op/s; 5.1 KiB/s, 1 objects/s recovering 2024-04-26T21:10:14.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:13 smithi204 ceph-mon[32145]: pgmap v385: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 545 B/s wr, 7 op/s; 5.1 KiB/s, 1 objects/s recovering 2024-04-26T21:10:14.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:13 smithi094 ceph-mon[31273]: pgmap v385: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 545 B/s wr, 7 op/s; 5.1 KiB/s, 1 objects/s recovering 2024-04-26T21:10:15.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 -- 172.21.15.27:0/1491672817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 msgr2=0x7f6b6010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 --2- 172.21.15.27:0/1491672817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6010ba40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009990 tx=0x7f6b50009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 -- 172.21.15.27:0/1491672817 shutdown_connections 2024-04-26T21:10:15.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 --2- 172.21.15.27:0/1491672817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 0x7f6b60112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 --2- 172.21.15.27:0/1491672817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6010ba40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 --2- 172.21.15.27:0/1491672817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b60105660 0x7f6b60105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 -- 172.21.15.27:0/1491672817 >> 172.21.15.27:0/1491672817 conn(0x7f6b60101150 msgr2=0x7f6b60103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 -- 172.21.15.27:0/1491672817 shutdown_connections 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.318+0000 7f6b67000700 1 -- 172.21.15.27:0/1491672817 wait complete. 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.319+0000 7f6b67000700 1 Processor -- start 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 -- start start 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b60105660 0x7f6b601a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 0x7f6b601a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60115740 con 0x7f6b6010b660 2024-04-26T21:10:15.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601155c0 con 0x7f6b60106020 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.320+0000 7f6b67000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601158c0 con 0x7f6b60105660 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53988/0 (socket says 172.21.15.27:53988) 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 -- 172.21.15.27:0/1954090216 learned_addr learned my addr 172.21.15.27:0/1954090216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b5fe00700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b60105660 0x7f6b601a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b5f400700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 0x7f6b601a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 -- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b60105660 msgr2=0x7f6b601a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b60105660 0x7f6b601a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 -- 172.21.15.27:0/1954090216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 msgr2=0x7f6b601a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 0x7f6b601a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b65000700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6019c200 con 0x7f6b6010b660 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.321+0000 7f6b5f400700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106020 0x7f6b601a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:15.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b65000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6019baa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007d70 tx=0x7f6b5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58016020 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b5800ad40 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b580115e0 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6019c4f0 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.322+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a8760 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.323+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b60063bb0 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.326+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b58019020 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5cc00700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b440a3d50 0x7f6b440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6b580d0830 con 0x7f6b6010b660 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5fe00700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b440a3d50 0x7f6b440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5cc00700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] conn(0x7f6b440ad910 0x7f6b440afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b58024e50 con 0x7f6b440ad910 2024-04-26T21:10:15.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.327+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6b580d0be0 con 0x7f6b6010b660 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.328+0000 7f6b5fe00700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b440a3d50 0x7f6b440a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b50003760 tx=0x7f6b5001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.341+0000 7f6b5f400700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] conn(0x7f6b440ad910 0x7f6b440afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.342+0000 7f6b5f400700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] conn(0x7f6b440ad910 0x7f6b440afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.345+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== osd.8 v2:172.21.15.204:6816/3522528239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b58024e50 con 0x7f6b440ad910 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.364+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 --> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b60064030 con 0x7f6b440ad910 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b5cc00700 1 -- 172.21.15.27:0/1954090216 <== osd.8 v2:172.21.15.204:6816/3522528239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b60064030 con 0x7f6b440ad910 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] conn(0x7f6b440ad910 msgr2=0x7f6b440afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] conn(0x7f6b440ad910 0x7f6b440afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b440a3d50 msgr2=0x7f6b440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6b440a3d50 0x7f6b440a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b50003760 tx=0x7f6b5001a040 comp rx=0 tx=0).stop 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 msgr2=0x7f6b6019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.366+0000 7f6b67000700 1 --2- 172.21.15.27:0/1954090216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b6010b660 0x7f6b6019baa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007d70 tx=0x7f6b5800a340 comp rx=0 tx=0).stop 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.367+0000 7f6b65000700 1 -- 172.21.15.27:0/1954090216 reap_dead start 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.367+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 shutdown_connections 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.367+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 >> 172.21.15.27:0/1954090216 conn(0x7f6b60101150 msgr2=0x7f6b60101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.367+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 shutdown_connections 2024-04-26T21:10:15.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:15.367+0000 7f6b67000700 1 -- 172.21.15.27:0/1954090216 wait complete. 2024-04-26T21:10:15.855 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:15.857 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:15.857 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:15.859 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:15.869 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:15.971 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:15 smithi027 ceph-mon[28085]: pgmap v386: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 493 B/s wr, 8 op/s; 4.6 KiB/s, 1 objects/s recovering 2024-04-26T21:10:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:15 smithi204 ceph-mon[32145]: pgmap v386: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 493 B/s wr, 8 op/s; 4.6 KiB/s, 1 objects/s recovering 2024-04-26T21:10:16.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:15 smithi094 ceph-mon[31273]: pgmap v386: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 493 B/s wr, 8 op/s; 4.6 KiB/s, 1 objects/s recovering 2024-04-26T21:10:17.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:17 smithi027 ceph-mon[28085]: pgmap v387: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:18.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:17 smithi204 ceph-mon[32145]: pgmap v387: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:17 smithi094 ceph-mon[31273]: pgmap v387: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:19 smithi204 ceph-mon[32145]: pgmap v388: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:20.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:19 smithi094 ceph-mon[31273]: pgmap v388: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:20.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:19 smithi027 ceph-mon[28085]: pgmap v388: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s; 3.9 KiB/s, 0 objects/s recovering 2024-04-26T21:10:20.319 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:21.238 INFO:teuthology.orchestra.run.smithi027.stdout:266287972396 2024-04-26T21:10:21.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T21:10:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 -- 172.21.15.27:0/167788854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f810e830 msgr2=0x7f53f810ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 --2- 172.21.15.27:0/167788854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f810e830 0x7f53f810ec10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009990 tx=0x7f53e8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 -- 172.21.15.27:0/167788854 shutdown_connections 2024-04-26T21:10:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 --2- 172.21.15.27:0/167788854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 0x7f53f81167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 --2- 172.21.15.27:0/167788854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f810e830 0x7f53f810ec10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 --2- 172.21.15.27:0/167788854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f8109960 0x7f53f8109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 -- 172.21.15.27:0/167788854 >> 172.21.15.27:0/167788854 conn(0x7f53f80791b0 msgr2=0x7f53f8108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 -- 172.21.15.27:0/167788854 shutdown_connections 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.806+0000 7f53fca00700 1 -- 172.21.15.27:0/167788854 wait complete. 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.807+0000 7f53fca00700 1 Processor -- start 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 -- start start 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 0x7f53f819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 0x7f53f81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f810e830 0x7f53f81a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8119a20 con 0x7f53f8109960 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81198a0 con 0x7f53f810a320 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53fca00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8119ba0 con 0x7f53f810e830 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53f7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 0x7f53f819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53f7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 0x7f53f819ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54810/0 (socket says 172.21.15.27:54810) 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.808+0000 7f53f7400700 1 -- 172.21.15.27:0/3399195652 learned_addr learned my addr 172.21.15.27:0/3399195652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f6a00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 0x7f53f81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 -- 172.21.15.27:0/3399195652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f810e830 msgr2=0x7f53f81a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f810e830 0x7f53f81a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 -- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 msgr2=0x7f53f81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 0x7f53f81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f81a4f20 con 0x7f53f8109960 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f6a00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53f810a320 0x7f53f81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53f7400700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 0x7f53f819ff10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.809+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8027070 con 0x7f53f8109960 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.810+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81a51b0 con 0x7f53f8109960 2024-04-26T21:10:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.810+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53e800fc80 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.810+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8022d60 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.810+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81aca90 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.810+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53f8063bb0 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.813+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53e802a070 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.814+0000 7f53efe00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f53dc0a3cd0 0x7f53dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.814+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f53e80d1920 con 0x7f53f8109960 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.814+0000 7f53f6a00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f53dc0a3cd0 0x7f53dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.815+0000 7f53efe00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] conn(0x7f53dc0ad890 0x7f53dc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.815+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53dc0b0400 con 0x7f53dc0ad890 2024-04-26T21:10:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.815+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f53e80d1cd0 con 0x7f53f8109960 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.815+0000 7f53f6a00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f53dc0a3cd0 0x7f53dc0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f53e4005fd0 tx=0x7f53e4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.815+0000 7f53f7e00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] conn(0x7f53dc0ad890 0x7f53dc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.816+0000 7f53f7e00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] conn(0x7f53dc0ad890 0x7f53dc0afd30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.819+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== osd.1 v2:172.21.15.94:6800/3294273778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53dc0b0400 con 0x7f53dc0ad890 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.837+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 --> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53f8063fb0 con 0x7f53dc0ad890 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.838+0000 7f53efe00700 1 -- 172.21.15.27:0/3399195652 <== osd.1 v2:172.21.15.94:6800/3294273778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53f8063fb0 con 0x7f53dc0ad890 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.838+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] conn(0x7f53dc0ad890 msgr2=0x7f53dc0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.838+0000 7f53fca00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] conn(0x7f53dc0ad890 0x7f53dc0afd30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f53dc0a3cd0 msgr2=0x7f53dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f53dc0a3cd0 0x7f53dc0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f53e4005fd0 tx=0x7f53e4005dc0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 msgr2=0x7f53f819ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 --2- 172.21.15.27:0/3399195652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53f8109960 0x7f53f819ff10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e80036f0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53f7e00700 1 -- 172.21.15.27:0/3399195652 reap_dead start 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 shutdown_connections 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 >> 172.21.15.27:0/3399195652 conn(0x7f53f80791b0 msgr2=0x7f53f8079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 shutdown_connections 2024-04-26T21:10:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:19.839+0000 7f53fca00700 1 -- 172.21.15.27:0/3399195652 wait complete. 2024-04-26T21:10:21.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:21 smithi027 ceph-mon[28085]: pgmap v389: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:21 smithi094 ceph-mon[31273]: pgmap v389: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 -- 172.21.15.27:0/4018241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec105600 msgr2=0x7f86ec105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 --2- 172.21.15.27:0/4018241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec105600 0x7f86ec105a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009990 tx=0x7f86d8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 -- 172.21.15.27:0/4018241095 shutdown_connections 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 --2- 172.21.15.27:0/4018241095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 0x7f86ec1124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 --2- 172.21.15.27:0/4018241095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec10b690 0x7f86ec10ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 --2- 172.21.15.27:0/4018241095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec105600 0x7f86ec105a80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.188+0000 7f86f2c00700 1 -- 172.21.15.27:0/4018241095 >> 172.21.15.27:0/4018241095 conn(0x7f86ec1010d0 msgr2=0x7f86ec1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.189+0000 7f86f2c00700 1 -- 172.21.15.27:0/4018241095 shutdown_connections 2024-04-26T21:10:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.189+0000 7f86f2c00700 1 -- 172.21.15.27:0/4018241095 wait complete. 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.190+0000 7f86f2c00700 1 Processor -- start 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.190+0000 7f86f2c00700 1 -- start start 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.190+0000 7f86f2c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec105600 0x7f86ec1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f2c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 0x7f86ec1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f2c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 0x7f86ec19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f2c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec115700 con 0x7f86ec10b690 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f2c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec115580 con 0x7f86ec105fc0 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f2c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec115880 con 0x7f86ec105600 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f0c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 0x7f86ec19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f0c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 0x7f86ec19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54834/0 (socket says 172.21.15.27:54834) 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86f0c00700 1 -- 172.21.15.27:0/477550610 learned_addr learned my addr 172.21.15.27:0/477550610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86ebe00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec105600 0x7f86ec1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.191+0000 7f86eb400700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 0x7f86ec1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 -- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec105600 msgr2=0x7f86ec1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec105600 0x7f86ec1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 -- 172.21.15.27:0/477550610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 msgr2=0x7f86ec1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 0x7f86ec1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec19c220 con 0x7f86ec10b690 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86eb400700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86ec105fc0 0x7f86ec1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86ebe00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86ec105600 0x7f86ec1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f0c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 0x7f86ec19bac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007ee0 tx=0x7f86e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e400dbd0 con 0x7f86ec10b690 2024-04-26T21:10:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86e400dd70 con 0x7f86ec10b690 2024-04-26T21:10:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec19c510 con 0x7f86ec10b690 2024-04-26T21:10:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.192+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e40058d0 con 0x7f86ec10b690 2024-04-26T21:10:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.193+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec1a87d0 con 0x7f86ec10b690 2024-04-26T21:10:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.193+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86ec063bb0 con 0x7f86ec10b690 2024-04-26T21:10:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.196+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86e401a020 con 0x7f86ec10b690 2024-04-26T21:10:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.197+0000 7f86e8c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f86d00a3d50 0x7f86d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.197+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f86e40d0440 con 0x7f86ec10b690 2024-04-26T21:10:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.198+0000 7f86ebe00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f86d00a3d50 0x7f86d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.198+0000 7f86e8c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] conn(0x7f86d00ad910 0x7f86d00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.198+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86d00b0480 con 0x7f86d00ad910 2024-04-26T21:10:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.198+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f86e40d07f0 con 0x7f86ec10b690 2024-04-26T21:10:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.198+0000 7f86ebe00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f86d00a3d50 0x7f86d00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005e10 tx=0x7f86dc005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.199+0000 7f86eb400700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] conn(0x7f86d00ad910 0x7f86d00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.200+0000 7f86eb400700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] conn(0x7f86d00ad910 0x7f86d00afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.202+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== osd.2 v2:172.21.15.204:6800/3559310933 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86d00b0480 con 0x7f86d00ad910 2024-04-26T21:10:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.221+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 --> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86ec063fb0 con 0x7f86d00ad910 2024-04-26T21:10:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.222+0000 7f86e8c00700 1 -- 172.21.15.27:0/477550610 <== osd.2 v2:172.21.15.204:6800/3559310933 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86ec063fb0 con 0x7f86d00ad910 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.222+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] conn(0x7f86d00ad910 msgr2=0x7f86d00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.222+0000 7f86f2c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] conn(0x7f86d00ad910 0x7f86d00afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.222+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f86d00a3d50 msgr2=0x7f86d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.222+0000 7f86f2c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f86d00a3d50 0x7f86d00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005e10 tx=0x7f86dc005d60 comp rx=0 tx=0).stop 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 msgr2=0x7f86ec19bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 --2- 172.21.15.27:0/477550610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86ec10b690 0x7f86ec19bac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007ee0 tx=0x7f86e400d3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f0c00700 1 -- 172.21.15.27:0/477550610 reap_dead start 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 shutdown_connections 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 >> 172.21.15.27:0/477550610 conn(0x7f86ec1010d0 msgr2=0x7f86ec1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 shutdown_connections 2024-04-26T21:10:22.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:22.223+0000 7f86f2c00700 1 -- 172.21.15.27:0/477550610 wait complete. 2024-04-26T21:10:22.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:21 smithi204 ceph-mon[32145]: pgmap v389: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:23.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:22 smithi027 ceph-mon[28085]: pgmap v390: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:23.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:22 smithi204 ceph-mon[32145]: pgmap v390: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:23.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:22 smithi094 ceph-mon[31273]: pgmap v390: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:23.959 INFO:teuthology.orchestra.run.smithi027.stdout:77309411414 2024-04-26T21:10:23.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T21:10:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:25 smithi027 ceph-mon[28085]: pgmap v391: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-26T21:10:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:25 smithi204 ceph-mon[32145]: pgmap v391: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-26T21:10:25.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:25 smithi094 ceph-mon[31273]: pgmap v391: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-26T21:10:27.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 -- 172.21.15.27:0/1084271966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 msgr2=0x7fdb5010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 --2- 172.21.15.27:0/1084271966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb5010fe30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009990 tx=0x7fdb40009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 -- 172.21.15.27:0/1084271966 shutdown_connections 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 --2- 172.21.15.27:0/1084271966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 0x7fdb50116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 --2- 172.21.15.27:0/1084271966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb5010fe30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 --2- 172.21.15.27:0/1084271966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 0x7fdb50109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 -- 172.21.15.27:0/1084271966 >> 172.21.15.27:0/1084271966 conn(0x7fdb50076bf0 msgr2=0x7fdb50108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 -- 172.21.15.27:0/1084271966 shutdown_connections 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.159+0000 7fdb56600700 1 -- 172.21.15.27:0/1084271966 wait complete. 2024-04-26T21:10:27.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.161+0000 7fdb56600700 1 Processor -- start 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.161+0000 7fdb56600700 1 -- start start 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.161+0000 7fdb56600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 0x7fdb501a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.161+0000 7fdb56600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 0x7fdb501a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb56600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb501a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb56600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50119b90 con 0x7fdb5010fa50 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb56600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50119a10 con 0x7fdb50109a50 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb56600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50119d10 con 0x7fdb5010a410 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb501a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb501a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54850/0 (socket says 172.21.15.27:54850) 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 -- 172.21.15.27:0/941554547 learned_addr learned my addr 172.21.15.27:0/941554547 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4ea00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 0x7fdb501a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4f400700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 0x7fdb501a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 -- 172.21.15.27:0/941554547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 msgr2=0x7fdb501a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 0x7fdb501a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 -- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 msgr2=0x7fdb501a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 0x7fdb501a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4fe00700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb501a5140 con 0x7fdb5010fa50 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4ea00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5010a410 0x7fdb501a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.162+0000 7fdb4f400700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50109a50 0x7fdb501a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb4fe00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb501a4a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b970 tx=0x7fdb4800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48019070 con 0x7fdb5010fa50 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb4800ce40 con 0x7fdb5010fa50 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb501a5430 con 0x7fdb5010fa50 2024-04-26T21:10:27.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48007370 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.163+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb501accb0 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.164+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb50063b30 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.167+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb4801c070 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.168+0000 7fdb47e00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb340a3cd0 0x7fdb340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.168+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fdb480d0550 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.168+0000 7fdb4f400700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb340a3cd0 0x7fdb340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.169+0000 7fdb47e00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] conn(0x7fdb340ad890 0x7fdb340afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.169+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb48024e50 con 0x7fdb340ad890 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.169+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdb480d0900 con 0x7fdb5010fa50 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.169+0000 7fdb4f400700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb340a3cd0 0x7fdb340a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdb40003760 tx=0x7fdb4001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.169+0000 7fdb4ea00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] conn(0x7fdb340ad890 0x7fdb340afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.170+0000 7fdb4ea00700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] conn(0x7fdb340ad890 0x7fdb340afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.172+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== osd.4 v2:172.21.15.94:6808/128441754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb48024e50 con 0x7fdb340ad890 2024-04-26T21:10:27.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.191+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 --> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb50063fb0 con 0x7fdb340ad890 2024-04-26T21:10:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.192+0000 7fdb47e00700 1 -- 172.21.15.27:0/941554547 <== osd.4 v2:172.21.15.94:6808/128441754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb50063fb0 con 0x7fdb340ad890 2024-04-26T21:10:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.192+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] conn(0x7fdb340ad890 msgr2=0x7fdb340afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.192+0000 7fdb56600700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] conn(0x7fdb340ad890 0x7fdb340afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.192+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb340a3cd0 msgr2=0x7fdb340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb340a3cd0 0x7fdb340a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdb40003760 tx=0x7fdb4001a040 comp rx=0 tx=0).stop 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 msgr2=0x7fdb501a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 --2- 172.21.15.27:0/941554547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb5010fa50 0x7fdb501a4a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b970 tx=0x7fdb4800bd30 comp rx=0 tx=0).stop 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb4fe00700 1 -- 172.21.15.27:0/941554547 reap_dead start 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 shutdown_connections 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 >> 172.21.15.27:0/941554547 conn(0x7fdb50076bf0 msgr2=0x7fdb500772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 shutdown_connections 2024-04-26T21:10:27.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:27.193+0000 7fdb56600700 1 -- 172.21.15.27:0/941554547 wait complete. 2024-04-26T21:10:27.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:27 smithi027 ceph-mon[28085]: pgmap v392: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:27.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:27 smithi204 ceph-mon[32145]: pgmap v392: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:27.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:27 smithi094 ceph-mon[31273]: pgmap v392: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 -- 172.21.15.27:0/2876133311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7084102ab0 msgr2=0x7f708410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 --2- 172.21.15.27:0/2876133311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7084102ab0 0x7f708410f770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7074009990 tx=0x7f7074009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 -- 172.21.15.27:0/2876133311 shutdown_connections 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 --2- 172.21.15.27:0/2876133311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f708410fcb0 0x7f70841120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 --2- 172.21.15.27:0/2876133311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7084102100 0x7f70841024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 --2- 172.21.15.27:0/2876133311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7084102ab0 0x7f708410f770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 -- 172.21.15.27:0/2876133311 >> 172.21.15.27:0/2876133311 conn(0x7f70840fdd40 msgr2=0x7f7084100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.834+0000 7f708b200700 1 -- 172.21.15.27:0/2876133311 shutdown_connections 2024-04-26T21:10:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.835+0000 7f708b200700 1 -- 172.21.15.27:0/2876133311 wait complete. 2024-04-26T21:10:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.836+0000 7f708b200700 1 Processor -- start 2024-04-26T21:10:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.836+0000 7f708b200700 1 -- start start 2024-04-26T21:10:28.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7084102100 0x7f708410c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f708410fcb0 0x7f708410cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 0x7f70841ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7084115690 con 0x7f708410d430 2024-04-26T21:10:28.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7084115510 con 0x7f708410fcb0 2024-04-26T21:10:28.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.837+0000 7f708b200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7084115810 con 0x7f7084102100 2024-04-26T21:10:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f7089e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7084102100 0x7f708410c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f708a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 0x7f70841ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f708a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 0x7f70841ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54862/0 (socket says 172.21.15.27:54862) 2024-04-26T21:10:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f708a800700 1 -- 172.21.15.27:0/1510980308 learned_addr learned my addr 172.21.15.27:0/1510980308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f7089400700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f708410fcb0 0x7f708410cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.838+0000 7f708a800700 1 -- 172.21.15.27:0/1510980308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7084102100 msgr2=0x7f708410c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f708a800700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7084102100 0x7f708410c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f708a800700 1 -- 172.21.15.27:0/1510980308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f708410fcb0 msgr2=0x7f708410cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f708a800700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f708410fcb0 0x7f708410cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f708a800700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70841ac8b0 con 0x7f708410d430 2024-04-26T21:10:28.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f708a800700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 0x7f70841ac370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f708000efc0 tx=0x7f708000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.839+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70800092e0 con 0x7f708410d430 2024-04-26T21:10:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.840+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7080004bb0 con 0x7f708410d430 2024-04-26T21:10:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.840+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080004020 con 0x7f708410d430 2024-04-26T21:10:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.840+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70841acb40 con 0x7f708410d430 2024-04-26T21:10:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.840+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70841ad000 con 0x7f708410d430 2024-04-26T21:10:28.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.841+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7084063b30 con 0x7f708410d430 2024-04-26T21:10:28.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.844+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f708001b030 con 0x7f708410d430 2024-04-26T21:10:28.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.845+0000 7f707ea00700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f706c0a3d50 0x7f706c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.846+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f70800d0960 con 0x7f708410d430 2024-04-26T21:10:28.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.846+0000 7f7089e00700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f706c0a3d50 0x7f706c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.846+0000 7f707ea00700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] conn(0x7f706c0ad910 0x7f706c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.846+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f706c0b0480 con 0x7f706c0ad910 2024-04-26T21:10:28.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.846+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f70800d0d10 con 0x7f708410d430 2024-04-26T21:10:28.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.847+0000 7f7089400700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] conn(0x7f706c0ad910 0x7f706c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.847+0000 7f7089e00700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f706c0a3d50 0x7f706c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7078009c60 tx=0x7f7078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.847+0000 7f7089400700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] conn(0x7f706c0ad910 0x7f706c0afdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.848+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== osd.6 v2:172.21.15.27:6818/1091396972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f706c0b0480 con 0x7f706c0ad910 2024-04-26T21:10:28.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.867+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 --> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7084063fb0 con 0x7f706c0ad910 2024-04-26T21:10:28.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.867+0000 7f707ea00700 1 -- 172.21.15.27:0/1510980308 <== osd.6 v2:172.21.15.27:6818/1091396972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7084063fb0 con 0x7f706c0ad910 2024-04-26T21:10:28.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] conn(0x7f706c0ad910 msgr2=0x7f706c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] conn(0x7f706c0ad910 0x7f706c0afdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f706c0a3d50 msgr2=0x7f706c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f706c0a3d50 0x7f706c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7078009c60 tx=0x7f7078009260 comp rx=0 tx=0).stop 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 msgr2=0x7f70841ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 --2- 172.21.15.27:0/1510980308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f708410d430 0x7f70841ac370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f708000efc0 tx=0x7f708000c3a0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708a800700 1 -- 172.21.15.27:0/1510980308 reap_dead start 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 shutdown_connections 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 >> 172.21.15.27:0/1510980308 conn(0x7f70840fdd40 msgr2=0x7f7084100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 shutdown_connections 2024-04-26T21:10:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:28.868+0000 7f708b200700 1 -- 172.21.15.27:0/1510980308 wait complete. 2024-04-26T21:10:29.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:29 smithi027 ceph-mon[28085]: pgmap v393: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:29 smithi204 ceph-mon[32145]: pgmap v393: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:29.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:29 smithi094 ceph-mon[31273]: pgmap v393: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-26T21:10:30.792 INFO:teuthology.orchestra.run.smithi027.stdout:146028888139 2024-04-26T21:10:30.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.046+0000 7f9804600700 1 -- 172.21.15.27:0/2008631752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075650 msgr2=0x7f97fc075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.046+0000 7f9804600700 1 --2- 172.21.15.27:0/2008631752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075650 0x7f97fc075a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009990 tx=0x7f97f0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 -- 172.21.15.27:0/2008631752 shutdown_connections 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 --2- 172.21.15.27:0/2008631752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc1145f0 0x7f97fc1169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 --2- 172.21.15.27:0/2008631752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075650 0x7f97fc075a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 --2- 172.21.15.27:0/2008631752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97fc075f70 0x7f97fc1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 -- 172.21.15.27:0/2008631752 >> 172.21.15.27:0/2008631752 conn(0x7f97fc07a580 msgr2=0x7f97fc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 -- 172.21.15.27:0/2008631752 shutdown_connections 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.047+0000 7f9804600700 1 -- 172.21.15.27:0/2008631752 wait complete. 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.048+0000 7f9804600700 1 Processor -- start 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.048+0000 7f9804600700 1 -- start start 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc075650 0x7f97fc1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 0x7f97fc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97fc1145f0 0x7f97fc1a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc119af0 con 0x7f97fc075f70 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc119970 con 0x7f97fc075650 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9804600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc119c70 con 0x7f97fc1145f0 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9801200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 0x7f97fc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9801200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 0x7f97fc1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45342/0 (socket says 172.21.15.27:45342) 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9801200700 1 -- 172.21.15.27:0/1781285152 learned_addr learned my addr 172.21.15.27:0/1781285152 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:31.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.049+0000 7f9801c00700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc075650 0x7f97fc1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 -- 172.21.15.27:0/1781285152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97fc1145f0 msgr2=0x7f97fc1a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97fc1145f0 0x7f97fc1a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 -- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc075650 msgr2=0x7f97fc1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97fc075650 0x7f97fc1a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc1a50b0 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9801200700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 0x7f97fc1a05e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97e800d980 tx=0x7f97e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8017070 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.050+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc1a53a0 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.051+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97e8004830 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.051+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80053a0 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.051+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1acc20 con 0x7f97fc075f70 2024-04-26T21:10:31.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.052+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97fc113f60 con 0x7f97fc075f70 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.054+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f97e8004d00 con 0x7f97fc075f70 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.055+0000 7f97fa000700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97e40a3cd0 0x7f97e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f97e80d0a20 con 0x7f97fc075f70 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f9801c00700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97e40a3cd0 0x7f97e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f97fa000700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] conn(0x7f97e40ad890 0x7f97e40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97e8014ec0 con 0x7f97e40ad890 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f97e80d0dd0 con 0x7f97fc075f70 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.056+0000 7f9801c00700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97e40a3cd0 0x7f97e40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b5c0 tx=0x7f97f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.057+0000 7f9802600700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] conn(0x7f97e40ad890 0x7f97e40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.058+0000 7f9802600700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] conn(0x7f97e40ad890 0x7f97e40afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.060+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== osd.7 v2:172.21.15.94:6816/1853319664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97e8014ec0 con 0x7f97e40ad890 2024-04-26T21:10:31.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.097+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 --> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97fc076d40 con 0x7f97e40ad890 2024-04-26T21:10:31.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.098+0000 7f97fa000700 1 -- 172.21.15.27:0/1781285152 <== osd.7 v2:172.21.15.94:6816/1853319664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f97fc076d40 con 0x7f97e40ad890 2024-04-26T21:10:31.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] conn(0x7f97e40ad890 msgr2=0x7f97e40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] conn(0x7f97e40ad890 0x7f97e40afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97e40a3cd0 msgr2=0x7f97e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97e40a3cd0 0x7f97e40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b5c0 tx=0x7f97f001a040 comp rx=0 tx=0).stop 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 msgr2=0x7f97fc1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 --2- 172.21.15.27:0/1781285152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97fc075f70 0x7f97fc1a05e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97e800d980 tx=0x7f97e800dd40 comp rx=0 tx=0).stop 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9802600700 1 -- 172.21.15.27:0/1781285152 reap_dead start 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 shutdown_connections 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.099+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 >> 172.21.15.27:0/1781285152 conn(0x7f97fc07a580 msgr2=0x7f97fc113500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.100+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 shutdown_connections 2024-04-26T21:10:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:31.100+0000 7f9804600700 1 -- 172.21.15.27:0/1781285152 wait complete. 2024-04-26T21:10:31.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:31 smithi027 ceph-mon[28085]: pgmap v394: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-04-26T21:10:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:31 smithi204 ceph-mon[32145]: pgmap v394: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-04-26T21:10:31.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:31 smithi094 ceph-mon[31273]: pgmap v394: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 -- 172.21.15.27:0/1314604797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c8103470 msgr2=0x7f30c81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 --2- 172.21.15.27:0/1314604797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c8103470 0x7f30c81038f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 -- 172.21.15.27:0/1314604797 shutdown_connections 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 --2- 172.21.15.27:0/1314604797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c8103e30 0x7f30c8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 --2- 172.21.15.27:0/1314604797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c81094c0 0x7f30c81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 --2- 172.21.15.27:0/1314604797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c8103470 0x7f30c81038f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 -- 172.21.15.27:0/1314604797 >> 172.21.15.27:0/1314604797 conn(0x7f30c807a580 msgr2=0x7f30c807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 -- 172.21.15.27:0/1314604797 shutdown_connections 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.932+0000 7f30cfa00700 1 -- 172.21.15.27:0/1314604797 wait complete. 2024-04-26T21:10:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.933+0000 7f30cfa00700 1 Processor -- start 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 -- start start 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8103470 0x7f30c81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c8103e30 0x7f30c81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 0x7f30c819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8113470 con 0x7f30c81094c0 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81132f0 con 0x7f30c8103470 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.934+0000 7f30cfa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81135f0 con 0x7f30c8103e30 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 0x7f30c819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 0x7f30c819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45300/0 (socket says 172.21.15.27:45300) 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cd000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8103470 0x7f30c81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 -- 172.21.15.27:0/4095431700 learned_addr learned my addr 172.21.15.27:0/4095431700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30c7e00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c8103e30 0x7f30c81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 -- 172.21.15.27:0/4095431700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c8103e30 msgr2=0x7f30c81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c8103e30 0x7f30c81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 -- 172.21.15.27:0/4095431700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8103470 msgr2=0x7f30c81a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8103470 0x7f30c81a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.935+0000 7f30cda00700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c819c250 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30cda00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 0x7f30c819baf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f30c0007ee0 tx=0x7f30c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c000ddd0 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c0004020 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c819c540 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0015660 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.936+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c81a8800 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.937+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30c8063b30 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.939+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30c00041d0 con 0x7f30c81094c0 2024-04-26T21:10:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30c5600700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30b00a3d50 0x7f30b00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f30c00d0a90 con 0x7f30c81094c0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30cd000700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30b00a3d50 0x7f30b00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30c5600700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] conn(0x7f30b00ad910 0x7f30b00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30b00b0480 con 0x7f30b00ad910 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.941+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f30c00d0e40 con 0x7f30c81094c0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.942+0000 7f30c7e00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] conn(0x7f30b00ad910 0x7f30b00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.942+0000 7f30cd000700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30b00a3d50 0x7f30b00a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009be0 tx=0x7f30bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.942+0000 7f30c7e00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] conn(0x7f30b00ad910 0x7f30b00afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.943+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== osd.9 v2:172.21.15.27:6826/2596688206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30b00b0480 con 0x7f30b00ad910 2024-04-26T21:10:31.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.961+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 --> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30c8063f30 con 0x7f30b00ad910 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30c5600700 1 -- 172.21.15.27:0/4095431700 <== osd.9 v2:172.21.15.27:6826/2596688206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30c8063f30 con 0x7f30b00ad910 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] conn(0x7f30b00ad910 msgr2=0x7f30b00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] conn(0x7f30b00ad910 0x7f30b00afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30b00a3d50 msgr2=0x7f30b00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30b00a3d50 0x7f30b00a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009be0 tx=0x7f30bc009260 comp rx=0 tx=0).stop 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 msgr2=0x7f30c819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cfa00700 1 --2- 172.21.15.27:0/4095431700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c81094c0 0x7f30c819baf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f30c0007ee0 tx=0x7f30c000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.962+0000 7f30cda00700 1 -- 172.21.15.27:0/4095431700 reap_dead start 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.963+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 shutdown_connections 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.963+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 >> 172.21.15.27:0/4095431700 conn(0x7f30c807a580 msgr2=0x7f30c810e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.963+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 shutdown_connections 2024-04-26T21:10:31.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:29.963+0000 7f30cfa00700 1 -- 172.21.15.27:0/4095431700 wait complete. 2024-04-26T21:10:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 -- 172.21.15.27:0/2622290455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 msgr2=0x7f0a4c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 --2- 172.21.15.27:0/2622290455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c10f940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009990 tx=0x7f0a38009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 -- 172.21.15.27:0/2622290455 shutdown_connections 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 --2- 172.21.15.27:0/2622290455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 0x7f0a4c112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 --2- 172.21.15.27:0/2622290455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 0x7f0a4c075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 --2- 172.21.15.27:0/2622290455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c10f940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 -- 172.21.15.27:0/2622290455 >> 172.21.15.27:0/2622290455 conn(0x7f0a4c07a580 msgr2=0x7f0a4c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 -- 172.21.15.27:0/2622290455 shutdown_connections 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.265+0000 7f0a53200700 1 -- 172.21.15.27:0/2622290455 wait complete. 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.266+0000 7f0a53200700 1 Processor -- start 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.266+0000 7f0a53200700 1 -- start start 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.266+0000 7f0a53200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 0x7f0a4c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a53200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a53200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 0x7f0a4c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a53200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c115760 con 0x7f0a4c075f70 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a53200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c1155e0 con 0x7f0a4c10fe80 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a53200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c1158e0 con 0x7f0a4c075650 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c1a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45368/0 (socket says 172.21.15.27:45368) 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 -- 172.21.15.27:0/1777207518 learned_addr learned my addr 172.21.15.27:0/1777207518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 -- 172.21.15.27:0/1777207518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 msgr2=0x7f0a4c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4be00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 0x7f0a4c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a51200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 0x7f0a4c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 0x7f0a4c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 -- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 msgr2=0x7f0a4c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 0x7f0a4c19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4b400700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c19c260 con 0x7f0a4c075f70 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a4be00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a4c075650 0x7f0a4c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.267+0000 7f0a51200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a4c10fe80 0x7f0a4c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a4b400700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c1a1e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a380036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38027070 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a3800fc80 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38022d30 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c19c4f0 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.268+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1a87c0 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.269+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a4c063bb0 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.272+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0a3802a070 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a48c00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0a300a3cd0 0x7f0a300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0a380d1930 con 0x7f0a4c075f70 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a4be00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0a300a3cd0 0x7f0a300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a48c00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] conn(0x7f0a300ad890 0x7f0a300afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a3801ae50 con 0x7f0a300ad890 2024-04-26T21:10:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.274+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0a380d1ce0 con 0x7f0a4c075f70 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.275+0000 7f0a4be00700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0a300a3cd0 0x7f0a300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c005fd0 tx=0x7f0a3c005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.275+0000 7f0a51200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] conn(0x7f0a300ad890 0x7f0a300afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.276+0000 7f0a51200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] conn(0x7f0a300ad890 0x7f0a300afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.278+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== osd.10 v2:172.21.15.94:6824/141255194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a3801ae50 con 0x7f0a300ad890 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.297+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 --> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a4c064030 con 0x7f0a300ad890 2024-04-26T21:10:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.298+0000 7f0a48c00700 1 -- 172.21.15.27:0/1777207518 <== osd.10 v2:172.21.15.94:6824/141255194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a4c064030 con 0x7f0a300ad890 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] conn(0x7f0a300ad890 msgr2=0x7f0a300afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] conn(0x7f0a300ad890 0x7f0a300afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0a300a3cd0 msgr2=0x7f0a300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0a300a3cd0 0x7f0a300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c005fd0 tx=0x7f0a3c005de0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 msgr2=0x7f0a4c1a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 --2- 172.21.15.27:0/1777207518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c075f70 0x7f0a4c1a1e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a380036f0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a51200700 1 -- 172.21.15.27:0/1777207518 reap_dead start 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 shutdown_connections 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.299+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 >> 172.21.15.27:0/1777207518 conn(0x7f0a4c07a580 msgr2=0x7f0a4c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.300+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 shutdown_connections 2024-04-26T21:10:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.300+0000 7f0a53200700 1 -- 172.21.15.27:0/1777207518 wait complete. 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 -- 172.21.15.27:0/2634481469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104ea0 msgr2=0x7f640c10d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 --2- 172.21.15.27:0/2634481469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104ea0 0x7f640c10d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009990 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 -- 172.21.15.27:0/2634481469 shutdown_connections 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 --2- 172.21.15.27:0/2634481469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c10d9b0 0x7f640c10fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 --2- 172.21.15.27:0/2634481469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640c104580 0x7f640c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 --2- 172.21.15.27:0/2634481469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104ea0 0x7f640c10d390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 -- 172.21.15.27:0/2634481469 >> 172.21.15.27:0/2634481469 conn(0x7f640c0fddd0 msgr2=0x7f640c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 -- 172.21.15.27:0/2634481469 shutdown_connections 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.561+0000 7f6412200700 1 -- 172.21.15.27:0/2634481469 wait complete. 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 Processor -- start 2024-04-26T21:10:32.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 -- start start 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 0x7f640c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640c104ea0 0x7f640c1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c10d9b0 0x7f640c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1134a0 con 0x7f640c104580 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f640c113320 con 0x7f640c10d9b0 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.563+0000 7f6412200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f640c113620 con 0x7f640c104ea0 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 0x7f640c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 0x7f640c1a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45316/0 (socket says 172.21.15.27:45316) 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 -- 172.21.15.27:0/2267914335 learned_addr learned my addr 172.21.15.27:0/2267914335 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f640be00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640c104ea0 0x7f640c1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 -- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640c104ea0 msgr2=0x7f640c1a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640c104ea0 0x7f640c1a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 -- 172.21.15.27:0/2267914335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c10d9b0 msgr2=0x7f640c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f640c10d9b0 0x7f640c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.564+0000 7f6410e00700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c19c180 con 0x7f640c104580 2024-04-26T21:10:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6410e00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 0x7f640c1a1860 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f640000ada0 tx=0x7f640000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400017020 con 0x7f640c104580 2024-04-26T21:10:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f640000ce10 con 0x7f640c104580 2024-04-26T21:10:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c19c470 con 0x7f640c104580 2024-04-26T21:10:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000054e0 con 0x7f640c104580 2024-04-26T21:10:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.565+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a8730 con 0x7f640c104580 2024-04-26T21:10:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.566+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f640c063b30 con 0x7f640c104580 2024-04-26T21:10:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.568+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6400004c40 con 0x7f640c104580 2024-04-26T21:10:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f6409600700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63f40a3d50 0x7f63f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f64000d0780 con 0x7f640c104580 2024-04-26T21:10:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f640be00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63f40a3d50 0x7f63f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f6409600700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] conn(0x7f63f40ad910 0x7f63f40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6400019e50 con 0x7f63f40ad910 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.570+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f64000d0b30 con 0x7f640c104580 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.571+0000 7f640be00700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63f40a3d50 0x7f63f40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f640c104200 tx=0x7f63f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.571+0000 7f6411800700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] conn(0x7f63f40ad910 0x7f63f40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.572+0000 7f6411800700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] conn(0x7f63f40ad910 0x7f63f40afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.574+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== osd.11 v2:172.21.15.204:6824/1542928815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6400019e50 con 0x7f63f40ad910 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.595+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 --> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f640c063f30 con 0x7f63f40ad910 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.596+0000 7f6409600700 1 -- 172.21.15.27:0/2267914335 <== osd.11 v2:172.21.15.204:6824/1542928815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f640c063f30 con 0x7f63f40ad910 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] conn(0x7f63f40ad910 msgr2=0x7f63f40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] conn(0x7f63f40ad910 0x7f63f40afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63f40a3d50 msgr2=0x7f63f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f63f40a3d50 0x7f63f40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f640c104200 tx=0x7f63f801a040 comp rx=0 tx=0).stop 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 msgr2=0x7f640c1a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 --2- 172.21.15.27:0/2267914335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f640c104580 0x7f640c1a1860 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f640000ada0 tx=0x7f640000c330 comp rx=0 tx=0).stop 2024-04-26T21:10:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6411800700 1 -- 172.21.15.27:0/2267914335 reap_dead start 2024-04-26T21:10:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 shutdown_connections 2024-04-26T21:10:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 >> 172.21.15.27:0/2267914335 conn(0x7f640c0fddd0 msgr2=0x7f640c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.597+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 shutdown_connections 2024-04-26T21:10:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:30.598+0000 7f6412200700 1 -- 172.21.15.27:0/2267914335 wait complete. 2024-04-26T21:10:33.397 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:33 smithi027 ceph-mon[28085]: pgmap v395: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.132+0000 7fcf99600700 1 -- 172.21.15.27:0/2175068811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 msgr2=0x7fcf94102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.132+0000 7fcf99600700 1 --2- 172.21.15.27:0/2175068811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf94102890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf840099c0 tx=0x7fcf84009cd0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 -- 172.21.15.27:0/2175068811 shutdown_connections 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 --2- 172.21.15.27:0/2175068811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf94107590 0x7fcf9410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 --2- 172.21.15.27:0/2175068811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf94102890 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 --2- 172.21.15.27:0/2175068811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf94102e60 0x7fcf94106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 -- 172.21.15.27:0/2175068811 >> 172.21.15.27:0/2175068811 conn(0x7fcf940fdd40 msgr2=0x7fcf94100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 -- 172.21.15.27:0/2175068811 shutdown_connections 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.133+0000 7fcf99600700 1 -- 172.21.15.27:0/2175068811 wait complete. 2024-04-26T21:10:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.134+0000 7fcf99600700 1 Processor -- start 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 -- start start 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf941a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf94102e60 0x7fcf941a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf94107590 0x7fcf9419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94112320 con 0x7fcf941024b0 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941121a0 con 0x7fcf94107590 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf99600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941124a0 con 0x7fcf94102e60 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf93e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf941a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf93e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf941a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45354/0 (socket says 172.21.15.27:45354) 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.135+0000 7fcf93e00700 1 -- 172.21.15.27:0/2525375508 learned_addr learned my addr 172.21.15.27:0/2525375508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf98c00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf94107590 0x7fcf9419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93400700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf94102e60 0x7fcf941a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 -- 172.21.15.27:0/2525375508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf94102e60 msgr2=0x7fcf941a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf94102e60 0x7fcf941a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 -- 172.21.15.27:0/2525375508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf94107590 msgr2=0x7fcf9419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf94107590 0x7fcf9419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9419c160 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.136+0000 7fcf93e00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf941a17f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf840058e0 tx=0x7fcf8400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.137+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84026070 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.137+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf8400fdf0 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.137+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004cd0 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.137+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9419c3f0 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.137+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a86c0 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.138+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf940ffb10 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.140+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf84029070 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf90c00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf780a3d50 0x7fcf780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf840d1580 con 0x7fcf941024b0 2024-04-26T21:10:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf93400700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf780a3d50 0x7fcf780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf90c00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] conn(0x7fcf780ad910 0x7fcf780afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf84022e50 con 0x7fcf780ad910 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.142+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcf840d19d0 con 0x7fcf941024b0 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.143+0000 7fcf98c00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] conn(0x7fcf780ad910 0x7fcf780afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.143+0000 7fcf93400700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf780a3d50 0x7fcf780a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf80005fd0 tx=0x7fcf8000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.143+0000 7fcf98c00700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] conn(0x7fcf780ad910 0x7fcf780afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.144+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== osd.3 v2:172.21.15.27:6810/2585963482 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf84022e50 con 0x7fcf780ad910 2024-04-26T21:10:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.163+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 --> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf94103990 con 0x7fcf780ad910 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf90c00700 1 -- 172.21.15.27:0/2525375508 <== osd.3 v2:172.21.15.27:6810/2585963482 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf94103990 con 0x7fcf780ad910 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] conn(0x7fcf780ad910 msgr2=0x7fcf780afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] conn(0x7fcf780ad910 0x7fcf780afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf780a3d50 msgr2=0x7fcf780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf780a3d50 0x7fcf780a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf80005fd0 tx=0x7fcf8000a560 comp rx=0 tx=0).stop 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 msgr2=0x7fcf941a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.164+0000 7fcf99600700 1 --2- 172.21.15.27:0/2525375508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf941024b0 0x7fcf941a17f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf840058e0 tx=0x7fcf8400f740 comp rx=0 tx=0).stop 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.165+0000 7fcf98c00700 1 -- 172.21.15.27:0/2525375508 reap_dead start 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.165+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 shutdown_connections 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.165+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 >> 172.21.15.27:0/2525375508 conn(0x7fcf940fdd40 msgr2=0x7fcf9406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.165+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 shutdown_connections 2024-04-26T21:10:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:32.165+0000 7fcf99600700 1 -- 172.21.15.27:0/2525375508 wait complete. 2024-04-26T21:10:33.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:33 smithi204 ceph-mon[32145]: pgmap v395: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:10:33.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:33 smithi094 ceph-mon[31273]: pgmap v395: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 -- 172.21.15.27:0/2133972320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb6010b620 msgr2=0x7fdb6010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 --2- 172.21.15.27:0/2133972320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb6010b620 0x7fdb6010ba00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdb540099c0 tx=0x7fdb54009cd0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 -- 172.21.15.27:0/2133972320 shutdown_connections 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 --2- 172.21.15.27:0/2133972320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb60105f30 0x7fdb601123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 --2- 172.21.15.27:0/2133972320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb6010b620 0x7fdb6010ba00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 --2- 172.21.15.27:0/2133972320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb60105570 0x7fdb601059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.573+0000 7fdb66600700 1 -- 172.21.15.27:0/2133972320 >> 172.21.15.27:0/2133972320 conn(0x7fdb60101040 msgr2=0x7fdb60103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.574+0000 7fdb66600700 1 -- 172.21.15.27:0/2133972320 shutdown_connections 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.574+0000 7fdb66600700 1 -- 172.21.15.27:0/2133972320 wait complete. 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 Processor -- start 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 -- start start 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 0x7fdb601a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb60105f30 0x7fdb601a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb6010b620 0x7fdb6019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60115640 con 0x7fdb60105570 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601154c0 con 0x7fdb60105f30 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.575+0000 7fdb66600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601157c0 con 0x7fdb6010b620 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 0x7fdb601a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 0x7fdb601a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45390/0 (socket says 172.21.15.27:45390) 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 -- 172.21.15.27:0/208786968 learned_addr learned my addr 172.21.15.27:0/208786968 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb5fe00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb60105f30 0x7fdb601a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65c00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb6010b620 0x7fdb6019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 -- 172.21.15.27:0/208786968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb6010b620 msgr2=0x7fdb6019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb6010b620 0x7fdb6019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 -- 172.21.15.27:0/208786968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb60105f30 msgr2=0x7fdb601a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb60105f30 0x7fdb601a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.576+0000 7fdb65200700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6019c080 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb65200700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 0x7fdb601a1710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b580 tx=0x7fdb5400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54027070 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb5400fdd0 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6019c310 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540173f0 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.577+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb601a85e0 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.578+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb60063bb0 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.580+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb5402a070 con 0x7fdb60105570 2024-04-26T21:10:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5d600700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb480a3d50 0x7fdb480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fdb540d1ed0 con 0x7fdb60105570 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5fe00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb480a3d50 0x7fdb480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5d600700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] conn(0x7fdb480ad910 0x7fdb480afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb480b0480 con 0x7fdb480ad910 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.582+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdb540d2320 con 0x7fdb60105570 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.583+0000 7fdb65c00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] conn(0x7fdb480ad910 0x7fdb480afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.583+0000 7fdb5fe00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb480a3d50 0x7fdb480a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005950 tx=0x7fdb4c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.583+0000 7fdb65c00700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] conn(0x7fdb480ad910 0x7fdb480afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.584+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== osd.0 v2:172.21.15.27:6802/530549701 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb480b0480 con 0x7fdb480ad910 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.603+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 --> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb60063fb0 con 0x7fdb480ad910 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.603+0000 7fdb5d600700 1 -- 172.21.15.27:0/208786968 <== osd.0 v2:172.21.15.27:6802/530549701 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb60063fb0 con 0x7fdb480ad910 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] conn(0x7fdb480ad910 msgr2=0x7fdb480afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] conn(0x7fdb480ad910 0x7fdb480afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb480a3d50 msgr2=0x7fdb480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb480a3d50 0x7fdb480a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005950 tx=0x7fdb4c0058e0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 msgr2=0x7fdb601a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb66600700 1 --2- 172.21.15.27:0/208786968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb60105570 0x7fdb601a1710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b580 tx=0x7fdb5400f510 comp rx=0 tx=0).stop 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.604+0000 7fdb65c00700 1 -- 172.21.15.27:0/208786968 reap_dead start 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.605+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 shutdown_connections 2024-04-26T21:10:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.605+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 >> 172.21.15.27:0/208786968 conn(0x7fdb60101040 msgr2=0x7fdb60102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.605+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 shutdown_connections 2024-04-26T21:10:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:33.605+0000 7fdb66600700 1 -- 172.21.15.27:0/208786968 wait complete. 2024-04-26T21:10:34.669 INFO:teuthology.orchestra.run.smithi027.stdout:300647710759 2024-04-26T21:10:34.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T21:10:35.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:35 smithi027 ceph-mon[28085]: pgmap v396: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-26T21:10:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:35 smithi204 ceph-mon[32145]: pgmap v396: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-26T21:10:35.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:35 smithi094 ceph-mon[31273]: pgmap v396: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-26T21:10:35.703 INFO:teuthology.orchestra.run.smithi027.stdout:55834574941 2024-04-26T21:10:35.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T21:10:36.130 INFO:teuthology.orchestra.run.smithi027.stdout:206158430269 2024-04-26T21:10:36.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T21:10:37.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:37 smithi027 ceph-mon[28085]: pgmap v397: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:10:37.478 INFO:teuthology.orchestra.run.smithi027.stdout:227633266741 2024-04-26T21:10:37.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T21:10:37.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:37 smithi204 ceph-mon[32145]: pgmap v397: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:10:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:37 smithi094 ceph-mon[31273]: pgmap v397: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:10:38.188 INFO:teuthology.orchestra.run.smithi027.stdout:111669149777 2024-04-26T21:10:38.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T21:10:38.460 INFO:teuthology.orchestra.run.smithi027.stdout:356482285595 2024-04-26T21:10:38.460 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T21:10:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 -- 172.21.15.27:0/1607438057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e4105660 msgr2=0x7fe4e4105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 --2- 172.21.15.27:0/1607438057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e4105660 0x7fe4e4105ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009990 tx=0x7fe4d0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 -- 172.21.15.27:0/1607438057 shutdown_connections 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 --2- 172.21.15.27:0/1607438057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4e4106020 0x7fe4e4112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 --2- 172.21.15.27:0/1607438057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e410b660 0x7fe4e410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 --2- 172.21.15.27:0/1607438057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e4105660 0x7fe4e4105ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 -- 172.21.15.27:0/1607438057 >> 172.21.15.27:0/1607438057 conn(0x7fe4e4101150 msgr2=0x7fe4e4103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 -- 172.21.15.27:0/1607438057 shutdown_connections 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.459+0000 7fe4ec200700 1 -- 172.21.15.27:0/1607438057 wait complete. 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.460+0000 7fe4ec200700 1 Processor -- start 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 -- start start 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e4105660 0x7fe4e41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4e4106020 0x7fe4e41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 0x7fe4e419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41157a0 con 0x7fe4e410b660 2024-04-26T21:10:38.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4115620 con 0x7fe4e4106020 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.461+0000 7fe4ec200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4115920 con 0x7fe4e4105660 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 0x7fe4e419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 0x7fe4e419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45406/0 (socket says 172.21.15.27:45406) 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 -- 172.21.15.27:0/3508909591 learned_addr learned my addr 172.21.15.27:0/3508909591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4e8e00700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4e4106020 0x7fe4e41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4e9800700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e4105660 0x7fe4e41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 -- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e4105660 msgr2=0x7fe4e41a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e4105660 0x7fe4e41a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 -- 172.21.15.27:0/3508909591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4e4106020 msgr2=0x7fe4e41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4e4106020 0x7fe4e41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.462+0000 7fe4ea200700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e419c260 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4ea200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 0x7fe4e419bb00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc007ee0 tx=0x7fe4dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00dd20 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4dc004020 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e419c550 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc015660 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.463+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41a8810 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.465+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4e4063bb0 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.468+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe4dc0041d0 con 0x7fe4e410b660 2024-04-26T21:10:38.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.470+0000 7fe4e2000700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe4cc0a3cd0 0x7fe4cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.470+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe4dc0cff50 con 0x7fe4e410b660 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.470+0000 7fe4e9800700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe4cc0a3cd0 0x7fe4cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.471+0000 7fe4e2000700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] conn(0x7fe4cc0ad890 0x7fe4cc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.471+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4cc0b0400 con 0x7fe4cc0ad890 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.471+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc0d2570 con 0x7fe4e410b660 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.471+0000 7fe4e8e00700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] conn(0x7fe4cc0ad890 0x7fe4cc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.471+0000 7fe4e9800700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe4cc0a3cd0 0x7fe4cc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80045b0 tx=0x7fe4d800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.472+0000 7fe4e8e00700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] conn(0x7fe4cc0ad890 0x7fe4cc0afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.474+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== osd.5 v2:172.21.15.204:6808/4134827726 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4cc0b0400 con 0x7fe4cc0ad890 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.509+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 --> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4e4064030 con 0x7fe4cc0ad890 2024-04-26T21:10:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.510+0000 7fe4e2000700 1 -- 172.21.15.27:0/3508909591 <== osd.5 v2:172.21.15.204:6808/4134827726 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4e4064030 con 0x7fe4cc0ad890 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.510+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] conn(0x7fe4cc0ad890 msgr2=0x7fe4cc0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.510+0000 7fe4ec200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] conn(0x7fe4cc0ad890 0x7fe4cc0afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.510+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe4cc0a3cd0 msgr2=0x7fe4cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.510+0000 7fe4ec200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe4cc0a3cd0 0x7fe4cc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80045b0 tx=0x7fe4d800b3c0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 msgr2=0x7fe4e419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 --2- 172.21.15.27:0/3508909591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4e410b660 0x7fe4e419bb00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc007ee0 tx=0x7fe4dc00d3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ea200700 1 -- 172.21.15.27:0/3508909591 reap_dead start 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 shutdown_connections 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 >> 172.21.15.27:0/3508909591 conn(0x7fe4e4101150 msgr2=0x7fe4e4101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 shutdown_connections 2024-04-26T21:10:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:38.511+0000 7fe4ec200700 1 -- 172.21.15.27:0/3508909591 wait complete. 2024-04-26T21:10:39.371 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:39 smithi027 ceph-mon[28085]: pgmap v398: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:10:39.446 INFO:teuthology.orchestra.run.smithi027.stdout:171798691910 2024-04-26T21:10:39.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T21:10:39.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:39 smithi204 ceph-mon[32145]: pgmap v398: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:10:39.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:39 smithi094 ceph-mon[31273]: pgmap v398: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:10:40.356 INFO:teuthology.orchestra.run.smithi027.stdout:34359738472 2024-04-26T21:10:40.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T21:10:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:40.610 INFO:teuthology.orchestra.run.smithi027.stdout:322122547233 2024-04-26T21:10:40.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T21:10:41.167 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.168 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.315 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:41 smithi027 ceph-mon[28085]: pgmap v399: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:41.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:41 smithi204 ceph-mon[32145]: pgmap v399: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:41.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:41 smithi094 ceph-mon[31273]: pgmap v399: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:10:43.293 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:43 smithi027 ceph-mon[28085]: pgmap v400: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-26T21:10:43.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:43 smithi204 ceph-mon[32145]: pgmap v400: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-26T21:10:43.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:43 smithi094 ceph-mon[31273]: pgmap v400: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-26T21:10:45.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:45 smithi027 ceph-mon[28085]: pgmap v401: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-26T21:10:45.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:45 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:45 smithi204 ceph-mon[32145]: pgmap v401: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-26T21:10:45.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:45 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:45.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:45 smithi094 ceph-mon[31273]: pgmap v401: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-26T21:10:45.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:45 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:47.131 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:46 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.132 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:46 smithi094 ceph-mon[31273]: pgmap v402: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-26T21:10:47.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:46 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:46 smithi027 ceph-mon[28085]: pgmap v402: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-26T21:10:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:46 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:46 smithi204 ceph-mon[32145]: pgmap v402: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-26T21:10:49.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:48 smithi027 ceph-mon[28085]: pgmap v403: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:10:49.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:48 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:48 smithi204 ceph-mon[32145]: pgmap v403: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:10:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:48 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:48 smithi094 ceph-mon[31273]: pgmap v403: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:10:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:48 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:10:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.877+0000 7f0137e00700 1 -- 172.21.15.27:0/3451630196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075650 msgr2=0x7f0130075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.877+0000 7f0137e00700 1 --2- 172.21.15.27:0/3451630196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075650 0x7f0130075a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f012000b0a0 tx=0x7f012000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 -- 172.21.15.27:0/3451630196 shutdown_connections 2024-04-26T21:10:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 --2- 172.21.15.27:0/3451630196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301145f0 0x7f01301169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 --2- 172.21.15.27:0/3451630196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075650 0x7f0130075a30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 --2- 172.21.15.27:0/3451630196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075f70 0x7f01301140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 -- 172.21.15.27:0/3451630196 >> 172.21.15.27:0/3451630196 conn(0x7f013007a580 msgr2=0x7f013007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 -- 172.21.15.27:0/3451630196 shutdown_connections 2024-04-26T21:10:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.878+0000 7f0137e00700 1 -- 172.21.15.27:0/3451630196 wait complete. 2024-04-26T21:10:50.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 Processor -- start 2024-04-26T21:10:50.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 -- start start 2024-04-26T21:10:50.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075650 0x7f01301a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:50.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:50.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301145f0 0x7f01301a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0130119b00 con 0x7f0130075f70 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0130119980 con 0x7f0130075650 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.879+0000 7f0137e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0130119c80 con 0x7f01301145f0 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56134/0 (socket says 172.21.15.27:56134) 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 -- 172.21.15.27:0/2924815873 learned_addr learned my addr 172.21.15.27:0/2924815873 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0135400700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075650 0x7f01301a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 -- 172.21.15.27:0/2924815873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301145f0 msgr2=0x7f01301a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301145f0 0x7f01301a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 -- 172.21.15.27:0/2924815873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075650 msgr2=0x7f01301a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075650 0x7f01301a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.880+0000 7f0134a00700 1 -- 172.21.15.27:0/2924815873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301a5020 con 0x7f0130075f70 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f0134a00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c920 tx=0x7f011c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c007b00 con 0x7f0130075f70 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01301a5310 con 0x7f0130075f70 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f011c004cc0 con 0x7f0130075f70 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c0054a0 con 0x7f0130075f70 2024-04-26T21:10:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.881+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301acb90 con 0x7f0130075f70 2024-04-26T21:10:50.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.883+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013006a110 con 0x7f0130075f70 2024-04-26T21:10:50.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.885+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f011c004750 con 0x7f0130075f70 2024-04-26T21:10:50.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.887+0000 7f012e000700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 0x7f01140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:50.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.887+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f011c0d01e0 con 0x7f0130075f70 2024-04-26T21:10:50.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.887+0000 7f0135400700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 0x7f01140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:50.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.888+0000 7f0135400700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 0x7f01140a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e60 tx=0x7f012000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:50.892+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f011c014030 con 0x7f0130075f70 2024-04-26T21:10:51.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.135+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f01301a1370 con 0x7f0130075f70 2024-04-26T21:10:51.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.136+0000 7f012e000700 1 -- 172.21.15.27:0/2924815873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01301a1370 con 0x7f0130075f70 2024-04-26T21:10:51.137 INFO:teuthology.orchestra.run.smithi027.stdout:55834574947 2024-04-26T21:10:51.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 msgr2=0x7f01140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:51.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 0x7f01140a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e60 tx=0x7f012000b7b0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 msgr2=0x7f01301a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:51.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c920 tx=0x7f011c00cce0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 shutdown_connections 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f01140a3d50 0x7f01140a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01301145f0 0x7f01301a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0130075650 0x7f01301a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 --2- 172.21.15.27:0/2924815873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0130075f70 0x7f01301a0550 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 >> 172.21.15.27:0/2924815873 conn(0x7f013007a580 msgr2=0x7f0130110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 shutdown_connections 2024-04-26T21:10:51.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:51.139+0000 7f0137e00700 1 -- 172.21.15.27:0/2924815873 wait complete. 2024-04-26T21:10:51.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:51 smithi027 ceph-mon[28085]: pgmap v404: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:51.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:51 smithi204 ceph-mon[32145]: pgmap v404: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:51.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:51 smithi094 ceph-mon[31273]: pgmap v404: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:10:52.222 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:52 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2924815873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:10:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:52 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2924815873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:10:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:52 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2924815873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:10:53.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:53 smithi027 ceph-mon[28085]: pgmap v405: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:10:53.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:53 smithi204 ceph-mon[32145]: pgmap v405: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:10:53.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:53 smithi094 ceph-mon[31273]: pgmap v405: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.866+0000 7fdb2dc00700 1 -- 172.21.15.27:0/3076010915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 msgr2=0x7fdb28112000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.866+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/3076010915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb28112000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000b0a0 tx=0x7fdb2000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 -- 172.21.15.27:0/3076010915 shutdown_connections 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/3076010915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb28112000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/3076010915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb28075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/3076010915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 0x7fdb2810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 -- 172.21.15.27:0/3076010915 >> 172.21.15.27:0/3076010915 conn(0x7fdb280feea0 msgr2=0x7fdb281012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 -- 172.21.15.27:0/3076010915 shutdown_connections 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.867+0000 7fdb2dc00700 1 -- 172.21.15.27:0/3076010915 wait complete. 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.868+0000 7fdb2dc00700 1 Processor -- start 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.868+0000 7fdb2dc00700 1 -- start start 2024-04-26T21:10:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb281a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 0x7fdb281a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28115650 con 0x7fdb2810fbf0 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281154d0 con 0x7fdb280757f0 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2dc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281157d0 con 0x7fdb28076110 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56142/0 (socket says 172.21.15.27:56142) 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb2d200700 1 -- 172.21.15.27:0/2656862473 learned_addr learned my addr 172.21.15.27:0/2656862473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb27e00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb281a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.869+0000 7fdb27400700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 0x7fdb281a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 -- 172.21.15.27:0/2656862473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 msgr2=0x7fdb281a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 0x7fdb281a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 -- 172.21.15.27:0/2656862473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 msgr2=0x7fdb281a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb281a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 -- 172.21.15.27:0/2656862473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2819c170 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb27e00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb281a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2d200700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006070 tx=0x7fdb20004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20021070 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb20007c00 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2819c400 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.870+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200047f0 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.871+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281a8720 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.872+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2806a110 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.874+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb20003a00 con 0x7fdb2810fbf0 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.875+0000 7fdb24c00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 0x7fdb0c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.875+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fdb2000f020 con 0x7fdb2810fbf0 2024-04-26T21:10:53.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.876+0000 7fdb27e00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 0x7fdb0c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:53.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.876+0000 7fdb27e00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 0x7fdb0c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005fd0 tx=0x7fdb18005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:53.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:53.881+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdb2002a020 con 0x7fdb2810fbf0 2024-04-26T21:10:54.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.121+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdb2802e490 con 0x7fdb2810fbf0 2024-04-26T21:10:54.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.122+0000 7fdb24c00700 1 -- 172.21.15.27:0/2656862473 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb2009e490 con 0x7fdb2810fbf0 2024-04-26T21:10:54.123 INFO:teuthology.orchestra.run.smithi027.stdout:146028888144 2024-04-26T21:10:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 msgr2=0x7fdb0c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 0x7fdb0c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005fd0 tx=0x7fdb18005f60 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 msgr2=0x7fdb2819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006070 tx=0x7fdb20004e00 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 shutdown_connections 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdb0c0a3cd0 0x7fdb0c0a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb2810fbf0 0x7fdb2819ba10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb280757f0 0x7fdb281a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 --2- 172.21.15.27:0/2656862473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb28076110 0x7fdb281a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 >> 172.21.15.27:0/2656862473 conn(0x7fdb280feea0 msgr2=0x7fdb280ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:54.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.125+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 shutdown_connections 2024-04-26T21:10:54.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.126+0000 7fdb2dc00700 1 -- 172.21.15.27:0/2656862473 wait complete. 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.335+0000 7faeff200700 1 -- 172.21.15.27:0/733391463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef81094a0 msgr2=0x7faef8109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.335+0000 7faeff200700 1 --2- 172.21.15.27:0/733391463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef81094a0 0x7faef8109880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faee8009990 tx=0x7faee8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 -- 172.21.15.27:0/733391463 shutdown_connections 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 --2- 172.21.15.27:0/733391463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806bad0 0x7faef81102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 --2- 172.21.15.27:0/733391463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef81094a0 0x7faef8109880 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 --2- 172.21.15.27:0/733391463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef806b130 0x7faef806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 -- 172.21.15.27:0/733391463 >> 172.21.15.27:0/733391463 conn(0x7faef80792d0 msgr2=0x7faef80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 -- 172.21.15.27:0/733391463 shutdown_connections 2024-04-26T21:10:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.336+0000 7faeff200700 1 -- 172.21.15.27:0/733391463 wait complete. 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 Processor -- start 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 -- start start 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 0x7faef81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef81094a0 0x7faef819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faef8113490 con 0x7faef806bad0 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faef8113310 con 0x7faef806b130 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.337+0000 7faeff200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faef8113610 con 0x7faef81094a0 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56186/0 (socket says 172.21.15.27:56186) 2024-04-26T21:10:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 -- 172.21.15.27:0/346968626 learned_addr learned my addr 172.21.15.27:0/346968626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faefd200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef81094a0 0x7faef819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7e00700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 0x7faef81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 -- 172.21.15.27:0/346968626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef81094a0 msgr2=0x7faef819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef81094a0 0x7faef819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 -- 172.21.15.27:0/346968626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 msgr2=0x7faef81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 0x7faef81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7400700 1 -- 172.21.15.27:0/346968626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef819c1d0 con 0x7faef806bad0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.338+0000 7faef7e00700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 0x7faef81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faef7400700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faee400c9b0 tx=0x7faee400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4007b00 con 0x7faef806bad0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faee4007ca0 con 0x7faef806bad0 2024-04-26T21:10:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4018660 con 0x7faef806bad0 2024-04-26T21:10:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef819c4c0 con 0x7faef806bad0 2024-04-26T21:10:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.339+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a8780 con 0x7faef806bad0 2024-04-26T21:10:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.341+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef806a110 con 0x7faef806bad0 2024-04-26T21:10:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.343+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faee400f450 con 0x7faef806bad0 2024-04-26T21:10:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.345+0000 7faef4c00700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 0x7faedc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.345+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7faee40d0630 con 0x7faef806bad0 2024-04-26T21:10:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.345+0000 7faef7e00700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 0x7faedc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.346+0000 7faef7e00700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 0x7faedc0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faee8003760 tx=0x7faee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:54.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.350+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faee4014030 con 0x7faef806bad0 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 -- 172.21.15.27:0/2659123122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 msgr2=0x7f2a04110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2659123122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a04110250 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b0d0 tx=0x7f29fc00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 -- 172.21.15.27:0/2659123122 shutdown_connections 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2659123122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a04110250 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2659123122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 0x7f2a04103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2659123122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a04107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 -- 172.21.15.27:0/2659123122 >> 172.21.15.27:0/2659123122 conn(0x7f2a040fee30 msgr2=0x7f2a04101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 -- 172.21.15.27:0/2659123122 shutdown_connections 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.227+0000 7f2a09c00700 1 -- 172.21.15.27:0/2659123122 wait complete. 2024-04-26T21:10:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.228+0000 7f2a09c00700 1 Processor -- start 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.228+0000 7f2a09c00700 1 -- start start 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 0x7f2a041a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a041a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04113420 con 0x7f2a04108680 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041132a0 con 0x7f2a041035a0 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041135a0 con 0x7f2a04103f50 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56170/0 (socket says 172.21.15.27:56170) 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a09200700 1 -- 172.21.15.27:0/2192565385 learned_addr learned my addr 172.21.15.27:0/2192565385 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a03e00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 0x7f2a041a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.229+0000 7f2a03400700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a041a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 -- 172.21.15.27:0/2192565385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 msgr2=0x7f2a041a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a041a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 -- 172.21.15.27:0/2192565385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 msgr2=0x7f2a041a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 0x7f2a041a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 -- 172.21.15.27:0/2192565385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0419c0c0 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a03400700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a041a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.230+0000 7f2a09200700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007d10 tx=0x7f29fc007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.231+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc01f070 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.231+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0419c350 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.231+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a041a8670 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.231+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29fc003e10 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.231+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc01b430 con 0x7f2a04108680 2024-04-26T21:10:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.232+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4005330 con 0x7f2a04108680 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.234+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f29fc022070 con 0x7f2a04108680 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.236+0000 7f2a00c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 0x7f29e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.236+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f29fc00f020 con 0x7f2a04108680 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.236+0000 7f2a03e00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 0x7f29e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.237+0000 7f2a03e00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 0x7f29e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a0419caa0 tx=0x7f29f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.238+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f29fc029020 con 0x7f2a04108680 2024-04-26T21:10:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.481+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f29e40058f0 con 0x7f2a04108680 2024-04-26T21:10:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.482+0000 7f2a00c00700 1 -- 172.21.15.27:0/2192565385 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29fc09e620 con 0x7f2a04108680 2024-04-26T21:10:54.483 INFO:teuthology.orchestra.run.smithi027.stdout:77309411420 2024-04-26T21:10:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 msgr2=0x7f29e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 0x7f29e80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a0419caa0 tx=0x7f29f400b410 comp rx=0 tx=0).stop 2024-04-26T21:10:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 msgr2=0x7f2a0419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007d10 tx=0x7f29fc007d40 comp rx=0 tx=0).stop 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 shutdown_connections 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f29e80a3cd0 0x7f29e80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a04108680 0x7f2a0419b960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a041035a0 0x7f2a041a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 --2- 172.21.15.27:0/2192565385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a04103f50 0x7f2a041a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.485+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 >> 172.21.15.27:0/2192565385 conn(0x7f2a040fee30 msgr2=0x7f2a040ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.486+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 shutdown_connections 2024-04-26T21:10:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.486+0000 7f2a09c00700 1 -- 172.21.15.27:0/2192565385 wait complete. 2024-04-26T21:10:54.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.594+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faef819cfd0 con 0x7faef806bad0 2024-04-26T21:10:54.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.595+0000 7faef4c00700 1 -- 172.21.15.27:0/346968626 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faef819cfd0 con 0x7faef806bad0 2024-04-26T21:10:54.596 INFO:teuthology.orchestra.run.smithi027.stdout:171798691913 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.597+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 msgr2=0x7faedc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.597+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 0x7faedc0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faee8003760 tx=0x7faee8005fb0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.597+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 msgr2=0x7faef81a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.597+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faee400c9b0 tx=0x7faee400cd70 comp rx=0 tx=0).stop 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 shutdown_connections 2024-04-26T21:10:54.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faedc0a3d50 0x7faedc0a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef81094a0 0x7faef819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faef806b130 0x7faef81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 --2- 172.21.15.27:0/346968626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faef806bad0 0x7faef81a1df0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 >> 172.21.15.27:0/346968626 conn(0x7faef80792d0 msgr2=0x7faef810e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.598+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 shutdown_connections 2024-04-26T21:10:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:54.599+0000 7faeff200700 1 -- 172.21.15.27:0/346968626 wait complete. 2024-04-26T21:10:54.666 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:54 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2656862473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:10:54.666 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:54 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2192565385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:10:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:54 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2656862473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:10:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:54 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2192565385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:10:55.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:54 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2656862473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:10:55.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:54 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2192565385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:10:55.621 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:55 smithi027 ceph-mon[28085]: pgmap v406: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:10:55.621 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:55 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/346968626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:10:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:55 smithi094 ceph-mon[31273]: pgmap v406: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:10:55.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:55 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/346968626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:10:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:55 smithi204 ceph-mon[32145]: pgmap v406: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:10:56.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:55 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/346968626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:10:56.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411420 for osd.2 2024-04-26T21:10:56.006 DEBUG:teuthology.parallel:result is None 2024-04-26T21:10:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 -- 172.21.15.27:0/3221956267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 msgr2=0x7f38a810fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 --2- 172.21.15.27:0/3221956267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a810fe30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3898009990 tx=0x7f3898009ca0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 -- 172.21.15.27:0/3221956267 shutdown_connections 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 --2- 172.21.15.27:0/3221956267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a810a410 0x7f38a8116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 --2- 172.21.15.27:0/3221956267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a810fe30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 --2- 172.21.15.27:0/3221956267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a8109a50 0x7f38a8109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 -- 172.21.15.27:0/3221956267 >> 172.21.15.27:0/3221956267 conn(0x7f38a8076bf0 msgr2=0x7f38a8108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.619+0000 7f38ad800700 1 -- 172.21.15.27:0/3221956267 shutdown_connections 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.620+0000 7f38ad800700 1 -- 172.21.15.27:0/3221956267 wait complete. 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 Processor -- start 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 -- start start 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a8109a50 0x7f38a819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a810a410 0x7f38a81a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8119b00 con 0x7f38a810fa50 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8119980 con 0x7f38a810a410 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.621+0000 7f38ad800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8119c80 con 0x7f38a8109a50 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56202/0 (socket says 172.21.15.27:56202) 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 -- 172.21.15.27:0/338395406 learned_addr learned my addr 172.21.15.27:0/338395406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a6a00700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a8109a50 0x7f38a819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a6000700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a810a410 0x7f38a81a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 -- 172.21.15.27:0/338395406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a8109a50 msgr2=0x7f38a819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a8109a50 0x7f38a819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 -- 172.21.15.27:0/338395406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a810a410 msgr2=0x7f38a81a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a810a410 0x7f38a81a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 -- 172.21.15.27:0/338395406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81a4f80 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.622+0000 7f38a7400700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38a000ada0 tx=0x7f38a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.623+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017020 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.623+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38a000ce10 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.623+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a81a5270 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.623+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0005440 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.623+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81acaf0 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.624+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a806a110 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.626+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f38a0004ad0 con 0x7f38a810fa50 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.628+0000 7f389f400700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 0x7f388c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.628+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f38a00d02c0 con 0x7f38a810fa50 2024-04-26T21:10:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.628+0000 7f38a6a00700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 0x7f388c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.629+0000 7f38a6a00700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 0x7f388c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f389800b5c0 tx=0x7f389801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.630+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f38a0021020 con 0x7f38a810fa50 2024-04-26T21:10:56.198 INFO:teuthology.orchestra.run.smithi027.stdout:300647710764 2024-04-26T21:10:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.874+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f38a802e490 con 0x7f38a810fa50 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.875+0000 7f389f400700 1 -- 172.21.15.27:0/338395406 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f38a009cb60 con 0x7f38a810fa50 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.878+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 msgr2=0x7f388c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.878+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 0x7f388c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f389800b5c0 tx=0x7f389801a040 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 msgr2=0x7f38a81a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f38a000ada0 tx=0x7f38a000c330 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 shutdown_connections 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f388c0a3d50 0x7f388c0a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a810fa50 0x7f38a81a4840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a8109a50 0x7f38a819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 --2- 172.21.15.27:0/338395406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a810a410 0x7f38a81a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 >> 172.21.15.27:0/338395406 conn(0x7f38a8076bf0 msgr2=0x7f38a80771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 shutdown_connections 2024-04-26T21:10:56.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:55.879+0000 7f38ad800700 1 -- 172.21.15.27:0/338395406 wait complete. 2024-04-26T21:10:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 -- 172.21.15.27:0/3250322694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 msgr2=0x7f4628116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 --2- 172.21.15.27:0/3250322694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f4628116920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f462000b0a0 tx=0x7f462000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 -- 172.21.15.27:0/3250322694 shutdown_connections 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 --2- 172.21.15.27:0/3250322694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f4628116920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 --2- 172.21.15.27:0/3250322694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 0x7f462810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 --2- 172.21.15.27:0/3250322694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 0x7f4628109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 -- 172.21.15.27:0/3250322694 >> 172.21.15.27:0/3250322694 conn(0x7f4628076bf0 msgr2=0x7f4628108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 -- 172.21.15.27:0/3250322694 shutdown_connections 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.256+0000 7f4630400700 1 -- 172.21.15.27:0/3250322694 wait complete. 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.257+0000 7f4630400700 1 Processor -- start 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 -- start start 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 0x7f46281a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 0x7f46281a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4628119b50 con 0x7f462810a410 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46281199d0 con 0x7f462810fa50 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f4630400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4628119cd0 con 0x7f4628109a50 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f462d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f462d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56216/0 (socket says 172.21.15.27:56216) 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.258+0000 7f462d000700 1 -- 172.21.15.27:0/2162076629 learned_addr learned my addr 172.21.15.27:0/2162076629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462e400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 0x7f46281a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462da00700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 0x7f46281a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 -- 172.21.15.27:0/2162076629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 msgr2=0x7f46281a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 0x7f46281a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 -- 172.21.15.27:0/2162076629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 msgr2=0x7f46281a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 0x7f46281a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 -- 172.21.15.27:0/2162076629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46281a50c0 con 0x7f462810a410 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f462d000700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f461400d980 tx=0x7f461400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.259+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614017070 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.260+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46281a53b0 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.260+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4614004830 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.260+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140053a0 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.260+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46281acc30 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.261+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462806a110 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.263+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f46140049e0 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.264+0000 7f4626000700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 0x7f46100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.264+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f46140d0500 con 0x7f462810a410 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.264+0000 7f462da00700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 0x7f46100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.265+0000 7f462da00700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 0x7f46100a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f461c005fd0 tx=0x7f461c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.268+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4614020020 con 0x7f462810a410 2024-04-26T21:10:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.527+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f462802e490 con 0x7f462810a410 2024-04-26T21:10:56.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.527+0000 7f4626000700 1 -- 172.21.15.27:0/2162076629 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f461409cd10 con 0x7f462810a410 2024-04-26T21:10:56.529 INFO:teuthology.orchestra.run.smithi027.stdout:206158430274 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.530+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 msgr2=0x7f46100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.530+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 0x7f46100a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f461c005fd0 tx=0x7f461c005ee0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.530+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 msgr2=0x7f46281a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.530+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f461400d980 tx=0x7f461400dd40 comp rx=0 tx=0).stop 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 shutdown_connections 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f46100a3d50 0x7f46100a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f462810fa50 0x7f46281a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4628109a50 0x7f46281a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 --2- 172.21.15.27:0/2162076629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f462810a410 0x7f46281a05f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 >> 172.21.15.27:0/2162076629 conn(0x7f4628076bf0 msgr2=0x7f46280772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 shutdown_connections 2024-04-26T21:10:56.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:56.531+0000 7f4630400700 1 -- 172.21.15.27:0/2162076629 wait complete. 2024-04-26T21:10:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:56 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/338395406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:10:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:56 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/338395406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:10:57.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:56 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/338395406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:10:57.285 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-04-26T21:10:57.285 DEBUG:teuthology.parallel:result is None 2024-04-26T21:10:57.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:57 smithi027 ceph-mon[28085]: pgmap v407: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:10:57.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:57 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2162076629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:10:57.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:57 smithi094 ceph-mon[31273]: pgmap v407: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:10:57.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:57 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2162076629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:10:58.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:57 smithi204 ceph-mon[32145]: pgmap v407: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:10:58.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:57 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2162076629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:10:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 -- 172.21.15.27:0/1215748080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884103f50 msgr2=0x7f9884107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 --2- 172.21.15.27:0/1215748080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884103f50 0x7f9884107fa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98700099c0 tx=0x7f9870009cd0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 -- 172.21.15.27:0/1215748080 shutdown_connections 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 --2- 172.21.15.27:0/1215748080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9884108680 0x7f9884110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 --2- 172.21.15.27:0/1215748080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98841035a0 0x7f9884103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 --2- 172.21.15.27:0/1215748080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884103f50 0x7f9884107fa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.534+0000 7f9883e00700 1 -- 172.21.15.27:0/1215748080 >> 172.21.15.27:0/1215748080 conn(0x7f98840fee70 msgr2=0x7f9884101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.535+0000 7f9883e00700 1 -- 172.21.15.27:0/1215748080 shutdown_connections 2024-04-26T21:10:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.535+0000 7f9883e00700 1 -- 172.21.15.27:0/1215748080 wait complete. 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 Processor -- start 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 -- start start 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98841035a0 0x7f988419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9884103f50 0x7f988419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98841133f0 con 0x7f9884108680 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9884113270 con 0x7f9884103f50 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.536+0000 7f9883e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9884113570 con 0x7f98841035a0 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56238/0 (socket says 172.21.15.27:56238) 2024-04-26T21:10:58.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 -- 172.21.15.27:0/253228109 learned_addr learned my addr 172.21.15.27:0/253228109 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:10:58.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9882a00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98841035a0 0x7f988419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9882000700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9884103f50 0x7f988419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 -- 172.21.15.27:0/253228109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98841035a0 msgr2=0x7f988419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98841035a0 0x7f988419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 -- 172.21.15.27:0/253228109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9884103f50 msgr2=0x7f988419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9884103f50 0x7f988419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.537+0000 7f9883400700 1 -- 172.21.15.27:0/253228109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988419a050 con 0x7f9884108680 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f9883400700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f987c007d70 tx=0x7f987c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c016020 con 0x7f9884108680 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f987c00ad40 con 0x7f9884108680 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988419a340 con 0x7f9884108680 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0115e0 con 0x7f9884108680 2024-04-26T21:10:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.538+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98841a6620 con 0x7f9884108680 2024-04-26T21:10:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.540+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9884104a80 con 0x7f9884108680 2024-04-26T21:10:58.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.542+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f987c019020 con 0x7f9884108680 2024-04-26T21:10:58.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.543+0000 7f987b400700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 0x7f98680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:58.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.543+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f987c0d0480 con 0x7f9884108680 2024-04-26T21:10:58.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.544+0000 7f9882a00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 0x7f98680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:58.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.544+0000 7f9882a00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 0x7f98680a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9874005950 tx=0x7f98740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:58.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.548+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f987c020020 con 0x7f9884108680 2024-04-26T21:10:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.791+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f988402e490 con 0x7f9884108680 2024-04-26T21:10:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.792+0000 7f987b400700 1 -- 172.21.15.27:0/253228109 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f987c09ce00 con 0x7f9884108680 2024-04-26T21:10:58.793 INFO:teuthology.orchestra.run.smithi027.stdout:266287972405 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.794+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 msgr2=0x7f98680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.794+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 0x7f98680a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9874005950 tx=0x7f98740058e0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.794+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 msgr2=0x7f9884199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.794+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f987c007d70 tx=0x7f987c00a340 comp rx=0 tx=0).stop 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 shutdown_connections 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f98680a3cd0 0x7f98680a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9884108680 0x7f9884199770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98841035a0 0x7f988419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 --2- 172.21.15.27:0/253228109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9884103f50 0x7f988419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 >> 172.21.15.27:0/253228109 conn(0x7f98840fee70 msgr2=0x7f9884101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 shutdown_connections 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:10:58.795+0000 7f9883e00700 1 -- 172.21.15.27:0/253228109 wait complete. 2024-04-26T21:10:59.653 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:59 smithi027 ceph-mon[28085]: pgmap v408: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:10:59.653 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:10:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/253228109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:11:00.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:59 smithi204 ceph-mon[32145]: pgmap v408: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:00.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:10:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/253228109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:11:00.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:59 smithi094 ceph-mon[31273]: pgmap v408: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:00.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:10:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/253228109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:11:00.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888139 got 146028888144 for osd.4 2024-04-26T21:11:00.371 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:00.675 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/529125638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:11:00.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710759 got 300647710764 for osd.9 2024-04-26T21:11:00.749 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:01.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/529125638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:11:01.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/529125638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.093+0000 7fbe20e00700 1 -- 172.21.15.27:0/1618116581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 msgr2=0x7fbe1c06b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.093+0000 7fbe20e00700 1 --2- 172.21.15.27:0/1618116581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c06b510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009990 tx=0x7fbe0c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 -- 172.21.15.27:0/1618116581 shutdown_connections 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 --2- 172.21.15.27:0/1618116581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c10fed0 0x7fbe1c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 --2- 172.21.15.27:0/1618116581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c06b510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 --2- 172.21.15.27:0/1618116581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c06ba50 0x7fbe1c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 -- 172.21.15.27:0/1618116581 >> 172.21.15.27:0/1618116581 conn(0x7fbe1c0792d0 msgr2=0x7fbe1c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 -- 172.21.15.27:0/1618116581 shutdown_connections 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.094+0000 7fbe20e00700 1 -- 172.21.15.27:0/1618116581 wait complete. 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.095+0000 7fbe20e00700 1 Processor -- start 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.095+0000 7fbe20e00700 1 -- start start 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.095+0000 7fbe20e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.095+0000 7fbe20e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c06ba50 0x7fbe1c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.095+0000 7fbe20e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c10fed0 0x7fbe1c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe20e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c115700 con 0x7fbe1c06b130 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe20e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c115580 con 0x7fbe1c06ba50 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe20e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c115880 con 0x7fbe1c10fed0 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49248/0 (socket says 172.21.15.27:49248) 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 -- 172.21.15.27:0/529125638 learned_addr learned my addr 172.21.15.27:0/529125638 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe1a800700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c10fed0 0x7fbe1c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 -- 172.21.15.27:0/529125638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c10fed0 msgr2=0x7fbe1c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c10fed0 0x7fbe1c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 -- 172.21.15.27:0/529125638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c06ba50 msgr2=0x7fbe1c19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c06ba50 0x7fbe1c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 -- 172.21.15.27:0/529125638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c1a0d20 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.096+0000 7fbe19e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c005910 tx=0x7fbe0c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.097+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c027070 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.097+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe0c00fc00 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.097+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c022d40 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.097+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c1a0fb0 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.097+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a8890 con 0x7fbe1c06b130 2024-04-26T21:11:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.098+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c06a110 con 0x7fbe1c06b130 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stdout:356482285601 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.100+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe0c02a070 con 0x7fbe1c06b130 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.101+0000 7fbe12a00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 0x7fbe000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.102+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbe0c0d1300 con 0x7fbe1c06b130 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.102+0000 7fbe19400700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 0x7fbe000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.103+0000 7fbe19400700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 0x7fbe000a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005950 tx=0x7fbe080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.106+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe0c05f060 con 0x7fbe1c06b130 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.350+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fbe1c02e490 con 0x7fbe1c06b130 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.351+0000 7fbe12a00700 1 -- 172.21.15.27:0/529125638 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbe0c013070 con 0x7fbe1c06b130 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 msgr2=0x7fbe000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 0x7fbe000a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005950 tx=0x7fbe080058e0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 msgr2=0x7fbe1c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c005910 tx=0x7fbe0c0047b0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 shutdown_connections 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbe000a3d50 0x7fbe000a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe1c10fed0 0x7fbe1c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe1c06b130 0x7fbe1c19bd10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 --2- 172.21.15.27:0/529125638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c06ba50 0x7fbe1c19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.354+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 >> 172.21.15.27:0/529125638 conn(0x7fbe1c0792d0 msgr2=0x7fbe1c111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.355+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 shutdown_connections 2024-04-26T21:11:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:00.355+0000 7fbe20e00700 1 -- 172.21.15.27:0/529125638 wait complete. 2024-04-26T21:11:01.601 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:01 smithi027 ceph-mon[28085]: pgmap v409: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:02.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:01 smithi204 ceph-mon[32145]: pgmap v409: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:02.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:01 smithi094 ceph-mon[31273]: pgmap v409: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:02.032 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691910 got 171798691913 for osd.5 2024-04-26T21:11:02.032 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:02.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972396 got 266287972405 for osd.8 2024-04-26T21:11:02.492 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:02.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285595 got 356482285601 for osd.11 2024-04-26T21:11:02.894 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:02.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 -- 172.21.15.27:0/1561424341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105fd0 msgr2=0x7fa7781124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 --2- 172.21.15.27:0/1561424341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105fd0 0x7fa7781124e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b0a0 tx=0x7fa77000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 -- 172.21.15.27:0/1561424341 shutdown_connections 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 --2- 172.21.15.27:0/1561424341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105fd0 0x7fa7781124e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 --2- 172.21.15.27:0/1561424341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 --2- 172.21.15.27:0/1561424341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105610 0x7fa778105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 -- 172.21.15.27:0/1561424341 >> 172.21.15.27:0/1561424341 conn(0x7fa778101100 msgr2=0x7fa778103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 -- 172.21.15.27:0/1561424341 shutdown_connections 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.689+0000 7fa77d200700 1 -- 172.21.15.27:0/1561424341 wait complete. 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.690+0000 7fa77d200700 1 Processor -- start 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 -- start start 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 0x7fa7781a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa778115710 con 0x7fa778105610 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa778115590 con 0x7fa778105fd0 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.691+0000 7fa77d200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa778115890 con 0x7fa77810b6a0 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49266/0 (socket says 172.21.15.27:49266) 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 -- 172.21.15.27:0/3143968745 learned_addr learned my addr 172.21.15.27:0/3143968745 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa776800700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775400700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 0x7fa7781a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 -- 172.21.15.27:0/3143968745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 msgr2=0x7fa77819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 -- 172.21.15.27:0/3143968745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 msgr2=0x7fa7781a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 0x7fa7781a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775e00700 1 -- 172.21.15.27:0/3143968745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819c0f0 con 0x7fa778105610 2024-04-26T21:11:02.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa776800700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.692+0000 7fa775400700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 0x7fa7781a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa775e00700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa76800c8f0 tx=0x7fa76800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768004220 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa768007d90 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819c3e0 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768003ca0 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.693+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a86a0 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.694+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77806a110 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.697+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa768007900 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.699+0000 7fa76ea00700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 0x7fa75c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.699+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa7680d0580 con 0x7fa778105610 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.700+0000 7fa775400700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 0x7fa75c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:02.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.701+0000 7fa775400700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 0x7fa75c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa764005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.702+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa768014030 con 0x7fa778105610 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.946+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa77802e490 con 0x7fa778105610 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.947+0000 7fa76ea00700 1 -- 172.21.15.27:0/3143968745 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa76809ce10 con 0x7fa778105610 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi027.stdout:227633266748 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 msgr2=0x7fa75c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 0x7fa75c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa764005ee0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 msgr2=0x7fa7781a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa76800c8f0 tx=0x7fa76800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 shutdown_connections 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fa75c0a3d50 0x7fa75c0a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa77810b6a0 0x7fa77819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa778105610 0x7fa7781a17d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.950+0000 7fa77d200700 1 --2- 172.21.15.27:0/3143968745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa778105fd0 0x7fa7781a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.951+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 >> 172.21.15.27:0/3143968745 conn(0x7fa778101100 msgr2=0x7fa778102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.951+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 shutdown_connections 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:02.951+0000 7fa77d200700 1 -- 172.21.15.27:0/3143968745 wait complete. 2024-04-26T21:11:03.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:03 smithi027 ceph-mon[28085]: pgmap v410: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:03.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:03 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3143968745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:11:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:03 smithi204 ceph-mon[32145]: pgmap v410: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:04.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:03 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3143968745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:11:04.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:03 smithi094 ceph-mon[31273]: pgmap v410: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:04.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:03 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3143968745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:11:04.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.067+0000 7faddcc00700 1 -- 172.21.15.27:0/4187289648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd810b630 msgr2=0x7fadd810ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:04.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.067+0000 7faddcc00700 1 --2- 172.21.15.27:0/4187289648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd810b630 0x7fadd810ba10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009990 tx=0x7fadc8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 -- 172.21.15.27:0/4187289648 shutdown_connections 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 --2- 172.21.15.27:0/4187289648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105ff0 0x7fadd8112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 --2- 172.21.15.27:0/4187289648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd810b630 0x7fadd810ba10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 --2- 172.21.15.27:0/4187289648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd8105630 0x7fadd8105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 -- 172.21.15.27:0/4187289648 >> 172.21.15.27:0/4187289648 conn(0x7fadd8101100 msgr2=0x7fadd8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 -- 172.21.15.27:0/4187289648 shutdown_connections 2024-04-26T21:11:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.068+0000 7faddcc00700 1 -- 172.21.15.27:0/4187289648 wait complete. 2024-04-26T21:11:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.069+0000 7faddcc00700 1 Processor -- start 2024-04-26T21:11:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.069+0000 7faddcc00700 1 -- start start 2024-04-26T21:11:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105630 0x7fadd81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd810b630 0x7fadd819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81156c0 con 0x7fadd8105ff0 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8115540 con 0x7fadd810b630 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7faddcc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8115840 con 0x7fadd8105630 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7fadd5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7fadd5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49280/0 (socket says 172.21.15.27:49280) 2024-04-26T21:11:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.070+0000 7fadd5400700 1 -- 172.21.15.27:0/3341752344 learned_addr learned my addr 172.21.15.27:0/3341752344 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd6800700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd810b630 0x7fadd819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5e00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105630 0x7fadd81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 -- 172.21.15.27:0/3341752344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105630 msgr2=0x7fadd81a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105630 0x7fadd81a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 -- 172.21.15.27:0/3341752344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd810b630 msgr2=0x7fadd819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd810b630 0x7fadd819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 -- 172.21.15.27:0/3341752344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd819c1c0 con 0x7fadd8105ff0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadd5400700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fadc400e9a0 tx=0x7fadc400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.071+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc400cbd0 con 0x7fadd8105ff0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.072+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc400cd70 con 0x7fadd8105ff0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.072+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4018680 con 0x7fadd8105ff0 2024-04-26T21:11:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.072+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd819c4b0 con 0x7fadd8105ff0 2024-04-26T21:11:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.072+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd81a8770 con 0x7fadd8105ff0 2024-04-26T21:11:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.073+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd806a110 con 0x7fadd8105ff0 2024-04-26T21:11:04.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.076+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadc4010950 con 0x7fadd8105ff0 2024-04-26T21:11:04.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.077+0000 7fadcea00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:04.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.077+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fadc40d0930 con 0x7fadd8105ff0 2024-04-26T21:11:04.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.078+0000 7fadd5e00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:04.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.078+0000 7fadd5e00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadc800b650 tx=0x7fadc801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:04.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.082+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fadc409d140 con 0x7fadd8105ff0 2024-04-26T21:11:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.333+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fadd802e490 con 0x7fadd8105ff0 2024-04-26T21:11:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.334+0000 7fadcea00700 1 -- 172.21.15.27:0/3341752344 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fadc409ca40 con 0x7fadd8105ff0 2024-04-26T21:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout:111669149783 2024-04-26T21:11:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 msgr2=0x7fadbc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadc800b650 tx=0x7fadc801a040 comp rx=0 tx=0).stop 2024-04-26T21:11:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 msgr2=0x7fadd81a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fadc400e9a0 tx=0x7fadc400ed60 comp rx=0 tx=0).stop 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 shutdown_connections 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd810b630 0x7fadd819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd8105630 0x7fadd81a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 --2- 172.21.15.27:0/3341752344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadd8105ff0 0x7fadd81a1de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 >> 172.21.15.27:0/3341752344 conn(0x7fadd8101100 msgr2=0x7fadd81018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 shutdown_connections 2024-04-26T21:11:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:04.338+0000 7faddcc00700 1 -- 172.21.15.27:0/3341752344 wait complete. 2024-04-26T21:11:04.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:04 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3341752344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:11:05.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:04 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3341752344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:11:05.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:04 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3341752344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:11:05.444 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430269 got 206158430274 for osd.6 2024-04-26T21:11:05.444 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:05.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:05 smithi027 ceph-mon[28085]: pgmap v411: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:06.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:05 smithi204 ceph-mon[32145]: pgmap v411: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:06.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:05 smithi094 ceph-mon[31273]: pgmap v411: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:06.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149777 got 111669149783 for osd.3 2024-04-26T21:11:06.315 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:06.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266741 got 227633266748 for osd.7 2024-04-26T21:11:06.730 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:07 smithi027 ceph-mon[28085]: pgmap v412: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:07 smithi204 ceph-mon[32145]: pgmap v412: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:08.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:07 smithi094 ceph-mon[31273]: pgmap v412: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.702+0000 7fb35dc00700 1 -- 172.21.15.27:0/2399038710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb35810fe50 msgr2=0x7fb358112240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2399038710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb35810fe50 0x7fb358112240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb35000b0a0 tx=0x7fb35000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 -- 172.21.15.27:0/2399038710 shutdown_connections 2024-04-26T21:11:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2399038710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb35810fe50 0x7fb358112240 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2399038710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb358075650 0x7fb358075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2399038710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb35810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 -- 172.21.15.27:0/2399038710 >> 172.21.15.27:0/2399038710 conn(0x7fb35807a4f0 msgr2=0x7fb35807a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:08.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 -- 172.21.15.27:0/2399038710 shutdown_connections 2024-04-26T21:11:08.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.703+0000 7fb35dc00700 1 -- 172.21.15.27:0/2399038710 wait complete. 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 Processor -- start 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 -- start start 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb3581a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb35810fe50 0x7fb35819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581156e0 con 0x7fb358075650 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb358115560 con 0x7fb35810fe50 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.704+0000 7fb35dc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb358115860 con 0x7fb358075f70 2024-04-26T21:11:08.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:08.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49290/0 (socket says 172.21.15.27:49290) 2024-04-26T21:11:08.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 -- 172.21.15.27:0/2348975711 learned_addr learned my addr 172.21.15.27:0/2348975711 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:08.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356000700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb3581a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:08.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb357400700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb35810fe50 0x7fb35819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 -- 172.21.15.27:0/2348975711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 msgr2=0x7fb3581a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb3581a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 -- 172.21.15.27:0/2348975711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb35810fe50 msgr2=0x7fb35819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb35810fe50 0x7fb35819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356a00700 1 -- 172.21.15.27:0/2348975711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35819c020 con 0x7fb358075650 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.705+0000 7fb356000700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb3581a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb356a00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c9b0 tx=0x7fb34800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348007b00 con 0x7fb358075650 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb348007ca0 con 0x7fb358075650 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35819c310 con 0x7fb358075650 2024-04-26T21:11:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348018660 con 0x7fb358075650 2024-04-26T21:11:08.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.706+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a85d0 con 0x7fb358075650 2024-04-26T21:11:08.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.708+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35806a110 con 0x7fb358075650 2024-04-26T21:11:08.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.710+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb34800f450 con 0x7fb358075650 2024-04-26T21:11:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.711+0000 7fb34f400700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 0x7fb33c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.711+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb3480d0630 con 0x7fb358075650 2024-04-26T21:11:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.712+0000 7fb356000700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 0x7fb33c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.713+0000 7fb356000700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 0x7fb33c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb358111970 tx=0x7fb34400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.716+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb348014030 con 0x7fb358075650 2024-04-26T21:11:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.974+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb3581a8860 con 0x7fb358075650 2024-04-26T21:11:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.975+0000 7fb34f400700 1 -- 172.21.15.27:0/2348975711 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb34809ce40 con 0x7fb358075650 2024-04-26T21:11:08.976 INFO:teuthology.orchestra.run.smithi027.stdout:322122547240 2024-04-26T21:11:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 msgr2=0x7fb33c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 0x7fb33c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb358111970 tx=0x7fb34400a680 comp rx=0 tx=0).stop 2024-04-26T21:11:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 msgr2=0x7fb3581a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c9b0 tx=0x7fb34800cd70 comp rx=0 tx=0).stop 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 shutdown_connections 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb33c0a3d50 0x7fb33c0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb35810fe50 0x7fb35819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb358075650 0x7fb3581a1700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 --2- 172.21.15.27:0/2348975711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb358075f70 0x7fb3581a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 >> 172.21.15.27:0/2348975711 conn(0x7fb35807a4f0 msgr2=0x7fb358100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 shutdown_connections 2024-04-26T21:11:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:08.978+0000 7fb35dc00700 1 -- 172.21.15.27:0/2348975711 wait complete. 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 -- 172.21.15.27:0/622668475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 msgr2=0x7f3eac1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/622668475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1140b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009990 tx=0x7f3e98009ca0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 -- 172.21.15.27:0/622668475 shutdown_connections 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/622668475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac1145f0 0x7f3eac1169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/622668475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac075650 0x7f3eac075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/622668475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1140b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 -- 172.21.15.27:0/622668475 >> 172.21.15.27:0/622668475 conn(0x7f3eac07a5d0 msgr2=0x7f3eac07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:09.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 -- 172.21.15.27:0/622668475 shutdown_connections 2024-04-26T21:11:09.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.256+0000 7f3eb1a00700 1 -- 172.21.15.27:0/622668475 wait complete. 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 Processor -- start 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 -- start start 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac075650 0x7f3eac1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 0x7f3eac1a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac119b50 con 0x7f3eac075f70 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1199d0 con 0x7f3eac1145f0 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.258+0000 7f3eb1a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac119cd0 con 0x7f3eac075650 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40678/0 (socket says 172.21.15.27:40678) 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 -- 172.21.15.27:0/1970229644 learned_addr learned my addr 172.21.15.27:0/1970229644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaaa00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac075650 0x7f3eac1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:09.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eab400700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 0x7f3eac1a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 -- 172.21.15.27:0/1970229644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac075650 msgr2=0x7f3eac1a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac075650 0x7f3eac1a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 -- 172.21.15.27:0/1970229644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 msgr2=0x7f3eac1a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 0x7f3eac1a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eaa000700 1 -- 172.21.15.27:0/1970229644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eac1a50f0 con 0x7f3eac075f70 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.259+0000 7f3eab400700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 0x7f3eac1a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3eaa000700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000c00 tx=0x7f3e980036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98027070 con 0x7f3eac075f70 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9800fc80 con 0x7f3eac075f70 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eac1a5380 con 0x7f3eac075f70 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98022d60 con 0x7f3eac075f70 2024-04-26T21:11:09.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.260+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eac1acc60 con 0x7f3eac075f70 2024-04-26T21:11:09.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.262+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eac06a110 con 0x7f3eac075f70 2024-04-26T21:11:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.265+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e9802a070 con 0x7f3eac075f70 2024-04-26T21:11:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.267+0000 7f3ea3400700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 0x7f3e900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.267+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3e980d14f0 con 0x7f3eac075f70 2024-04-26T21:11:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.267+0000 7f3eaaa00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 0x7f3e900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:09.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.268+0000 7f3eaaa00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 0x7f3e900a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:09.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.271+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e9805f060 con 0x7f3eac075f70 2024-04-26T21:11:09.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.508+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3eac1142d0 con 0x7f3eac075f70 2024-04-26T21:11:09.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.508+0000 7f3ea3400700 1 -- 172.21.15.27:0/1970229644 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3eac1142d0 con 0x7f3eac075f70 2024-04-26T21:11:09.510 INFO:teuthology.orchestra.run.smithi027.stdout:34359738479 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 msgr2=0x7f3e900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 0x7f3e900a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c00a560 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 msgr2=0x7f3eac1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000c00 tx=0x7f3e980036f0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 shutdown_connections 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f3e900a3cd0 0x7f3e900a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eac1145f0 0x7f3eac1a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eac075650 0x7f3eac1a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 --2- 172.21.15.27:0/1970229644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3eac075f70 0x7f3eac1a0620 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.511+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 >> 172.21.15.27:0/1970229644 conn(0x7f3eac07a5d0 msgr2=0x7f3eac111410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.512+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 shutdown_connections 2024-04-26T21:11:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:09.512+0000 7f3eb1a00700 1 -- 172.21.15.27:0/1970229644 wait complete. 2024-04-26T21:11:09.886 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:09 smithi027 ceph-mon[28085]: pgmap v413: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:09.886 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2348975711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:11:09.886 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1970229644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:11:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:09 smithi204 ceph-mon[32145]: pgmap v413: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:09 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2348975711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:11:10.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:09 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1970229644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:11:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:09 smithi094 ceph-mon[31273]: pgmap v413: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:09 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2348975711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:11:10.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:09 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1970229644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:11:10.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738472 got 34359738479 for osd.0 2024-04-26T21:11:10.299 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:10.645 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547233 got 322122547240 for osd.10 2024-04-26T21:11:10.645 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:10.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T21:11:10.645 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:11:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:11.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:11 smithi027 ceph-mon[28085]: pgmap v414: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:11.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:11.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:11 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:11 smithi204 ceph-mon[32145]: pgmap v414: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:12.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:11 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:11 smithi094 ceph-mon[31273]: pgmap v414: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:11:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:12.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:11 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:12.880 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:12 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:12.880 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:12 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:13.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:12 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:13.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:12 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:13.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:12 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:13.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:12 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 -- 172.21.15.27:0/347254462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca41035a0 msgr2=0x7fbca4103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 --2- 172.21.15.27:0/347254462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca41035a0 0x7fbca4103980 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc940099c0 tx=0x7fbc94009cd0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 -- 172.21.15.27:0/347254462 shutdown_connections 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 --2- 172.21.15.27:0/347254462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca4108680 0x7fbca4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 --2- 172.21.15.27:0/347254462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca41035a0 0x7fbca4103980 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 --2- 172.21.15.27:0/347254462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 0x7fbca4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 -- 172.21.15.27:0/347254462 >> 172.21.15.27:0/347254462 conn(0x7fbca40fee30 msgr2=0x7fbca4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.195+0000 7fbca9600700 1 -- 172.21.15.27:0/347254462 shutdown_connections 2024-04-26T21:11:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.196+0000 7fbca9600700 1 -- 172.21.15.27:0/347254462 wait complete. 2024-04-26T21:11:13.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 Processor -- start 2024-04-26T21:11:13.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 -- start start 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca41035a0 0x7fbca4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 0x7fbca4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4113440 con 0x7fbca4108680 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41132c0 con 0x7fbca4103f50 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.197+0000 7fbca9600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41135c0 con 0x7fbca41035a0 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40692/0 (socket says 172.21.15.27:40692) 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 -- 172.21.15.27:0/2430870484 learned_addr learned my addr 172.21.15.27:0/2430870484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca3400700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 0x7fbca4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:13.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca3e00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca41035a0 0x7fbca4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:13.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 -- 172.21.15.27:0/2430870484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca41035a0 msgr2=0x7fbca4075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:13.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca41035a0 0x7fbca4075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 -- 172.21.15.27:0/2430870484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 msgr2=0x7fbca4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:13.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 0x7fbca4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.198+0000 7fbca8c00700 1 -- 172.21.15.27:0/2430870484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca41a7f00 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca8c00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c007ee0 tx=0x7fbc9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004bb0 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc9c004d50 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41a81f0 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0058b0 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.199+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a8730 con 0x7fbca4108680 2024-04-26T21:11:13.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.200+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca406a110 con 0x7fbca4108680 2024-04-26T21:11:13.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.203+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc9c01a020 con 0x7fbca4108680 2024-04-26T21:11:13.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.204+0000 7fbca0c00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 0x7fbc880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:13.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.204+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbc9c0cfc30 con 0x7fbca4108680 2024-04-26T21:11:13.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.204+0000 7fbca3e00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 0x7fbc880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:13.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.205+0000 7fbca3e00700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 0x7fbc880a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc94000c00 tx=0x7fbc94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:13.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.208+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbc9c010070 con 0x7fbca4108680 2024-04-26T21:11:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.440+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbca4063860 con 0x7fbc880a3cd0 2024-04-26T21:11:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.462+0000 7fbca0c00700 1 -- 172.21.15.27:0/2430870484 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258779 (secure 0 0 0) 0x7fbc940672f0 con 0x7fbc880a3cd0 2024-04-26T21:11:13.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.468+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 msgr2=0x7fbc880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.468+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 0x7fbc880a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc94000c00 tx=0x7fbc94005fb0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.468+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 msgr2=0x7fbca40761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.468+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c007ee0 tx=0x7fbc9c00d3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 shutdown_connections 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbc880a3cd0 0x7fbc880a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca4108680 0x7fbca40761a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbca41035a0 0x7fbca4075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 --2- 172.21.15.27:0/2430870484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbca4103f50 0x7fbca4075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:13.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.469+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 >> 172.21.15.27:0/2430870484 conn(0x7fbca40fee30 msgr2=0x7fbca41001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.470+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 shutdown_connections 2024-04-26T21:11:13.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:13.470+0000 7fbca9600700 1 -- 172.21.15.27:0/2430870484 wait complete. 2024-04-26T21:11:13.473 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T21:11:13.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:13 smithi027 ceph-mon[28085]: pgmap v415: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:13.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:13 smithi027 ceph-mon[28085]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:14.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:13 smithi204 ceph-mon[32145]: pgmap v415: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:14.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:13 smithi204 ceph-mon[32145]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:14.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:13 smithi094 ceph-mon[31273]: pgmap v415: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:11:14.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:13 smithi094 ceph-mon[31273]: from='mgr.14152 172.21.15.27:0/2227054871' entity='mgr.x' 2024-04-26T21:11:14.443 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-04-26T21:11:12.081631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650797,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":969,"num_read_kb":594,"num_write":421,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1229691,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396268,"kb_used_data":13764,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545556,"statfs":{"total":1151940427776,"available":1151534649344,"internally_reserved":0,"allocated":14094336,"data_stored":7588071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":65,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004204"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697923+0000","last_change":"2024-04-26T21:09:41.072073+0000","last_active":"2024-04-26T21:10:03.697923+0000","last_peered":"2024-04-26T21:10:03.697923+0000","last_clean":"2024-04-26T21:10:03.697923+0000","last_became_active":"2024-04-26T21:09:41.071687+0000","last_became_peered":"2024-04-26T21:09:41.071687+0000","last_unstale":"2024-04-26T21:10:03.697923+0000","last_undegraded":"2024-04-26T21:10:03.697923+0000","last_fullsized":"2024-04-26T21:10:03.697923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:17.092134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716245+0000","last_change":"2024-04-26T21:09:41.063568+0000","last_active":"2024-04-26T21:10:03.716245+0000","last_peered":"2024-04-26T21:10:03.716245+0000","last_clean":"2024-04-26T21:10:03.716245+0000","last_became_active":"2024-04-26T21:09:41.061712+0000","last_became_peered":"2024-04-26T21:09:41.061712+0000","last_unstale":"2024-04-26T21:10:03.716245+0000","last_undegraded":"2024-04-26T21:10:03.716245+0000","last_fullsized":"2024-04-26T21:10:03.716245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:38.894168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716760+0000","last_change":"2024-04-26T21:09:41.068164+0000","last_active":"2024-04-26T21:10:03.716760+0000","last_peered":"2024-04-26T21:10:03.716760+0000","last_clean":"2024-04-26T21:10:03.716760+0000","last_became_active":"2024-04-26T21:09:41.067074+0000","last_became_peered":"2024-04-26T21:09:41.067074+0000","last_unstale":"2024-04-26T21:10:03.716760+0000","last_undegraded":"2024-04-26T21:10:03.716760+0000","last_fullsized":"2024-04-26T21:10:03.716760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.900193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.714910+0000","last_change":"2024-04-26T21:10:03.713993+0000","last_active":"2024-04-26T21:10:03.714910+0000","last_peered":"2024-04-26T21:10:03.714910+0000","last_clean":"2024-04-26T21:10:03.714910+0000","last_became_active":"2024-04-26T21:10:03.713380+0000","last_became_peered":"2024-04-26T21:10:03.713380+0000","last_unstale":"2024-04-26T21:10:03.714910+0000","last_undegraded":"2024-04-26T21:10:03.714910+0000","last_fullsized":"2024-04-26T21:10:03.714910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:43.726445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707897+0000","last_change":"2024-04-26T21:09:41.070626+0000","last_active":"2024-04-26T21:10:03.707897+0000","last_peered":"2024-04-26T21:10:03.707897+0000","last_clean":"2024-04-26T21:10:03.707897+0000","last_became_active":"2024-04-26T21:09:41.070258+0000","last_became_peered":"2024-04-26T21:09:41.070258+0000","last_unstale":"2024-04-26T21:10:03.707897+0000","last_undegraded":"2024-04-26T21:10:03.707897+0000","last_fullsized":"2024-04-26T21:10:03.707897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:10.491491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.714363+0000","last_change":"2024-04-26T21:10:03.714363+0000","last_active":"2024-04-26T21:10:03.714363+0000","last_peered":"2024-04-26T21:10:03.714363+0000","last_clean":"2024-04-26T21:10:03.714363+0000","last_became_active":"2024-04-26T21:10:03.713233+0000","last_became_peered":"2024-04-26T21:10:03.713233+0000","last_unstale":"2024-04-26T21:10:03.714363+0000","last_undegraded":"2024-04-26T21:10:03.714363+0000","last_fullsized":"2024-04-26T21:10:03.714363+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.010890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690314+0000","last_change":"2024-04-26T21:09:41.069684+0000","last_active":"2024-04-26T21:10:03.690314+0000","last_peered":"2024-04-26T21:10:03.690314+0000","last_clean":"2024-04-26T21:10:03.690314+0000","last_became_active":"2024-04-26T21:09:41.069323+0000","last_became_peered":"2024-04-26T21:09:41.069323+0000","last_unstale":"2024-04-26T21:10:03.690314+0000","last_undegraded":"2024-04-26T21:10:03.690314+0000","last_fullsized":"2024-04-26T21:10:03.690314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.122606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.695833+0000","last_change":"2024-04-26T21:09:41.069130+0000","last_active":"2024-04-26T21:10:03.695833+0000","last_peered":"2024-04-26T21:10:03.695833+0000","last_clean":"2024-04-26T21:10:03.695833+0000","last_became_active":"2024-04-26T21:09:41.068682+0000","last_became_peered":"2024-04-26T21:09:41.068682+0000","last_unstale":"2024-04-26T21:10:03.695833+0000","last_undegraded":"2024-04-26T21:10:03.695833+0000","last_fullsized":"2024-04-26T21:10:03.695833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:39.389562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699230+0000","last_change":"2024-04-26T21:09:41.073930+0000","last_active":"2024-04-26T21:10:03.699230+0000","last_peered":"2024-04-26T21:10:03.699230+0000","last_clean":"2024-04-26T21:10:03.699230+0000","last_became_active":"2024-04-26T21:09:41.072566+0000","last_became_peered":"2024-04-26T21:09:41.072566+0000","last_unstale":"2024-04-26T21:10:03.699230+0000","last_undegraded":"2024-04-26T21:10:03.699230+0000","last_fullsized":"2024-04-26T21:10:03.699230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:48.613310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700662+0000","last_change":"2024-04-26T21:09:41.065598+0000","last_active":"2024-04-26T21:10:03.700662+0000","last_peered":"2024-04-26T21:10:03.700662+0000","last_clean":"2024-04-26T21:10:03.700662+0000","last_became_active":"2024-04-26T21:09:41.064977+0000","last_became_peered":"2024-04-26T21:09:41.064977+0000","last_unstale":"2024-04-26T21:10:03.700662+0000","last_undegraded":"2024-04-26T21:10:03.700662+0000","last_fullsized":"2024-04-26T21:10:03.700662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:39.776967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690907+0000","last_change":"2024-04-26T21:09:41.077939+0000","last_active":"2024-04-26T21:10:03.690907+0000","last_peered":"2024-04-26T21:10:03.690907+0000","last_clean":"2024-04-26T21:10:03.690907+0000","last_became_active":"2024-04-26T21:09:41.077641+0000","last_became_peered":"2024-04-26T21:09:41.077641+0000","last_unstale":"2024-04-26T21:10:03.690907+0000","last_undegraded":"2024-04-26T21:10:03.690907+0000","last_fullsized":"2024-04-26T21:10:03.690907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:47.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719794+0000","last_change":"2024-04-26T21:09:41.075109+0000","last_active":"2024-04-26T21:10:03.719794+0000","last_peered":"2024-04-26T21:10:03.719794+0000","last_clean":"2024-04-26T21:10:03.719794+0000","last_became_active":"2024-04-26T21:09:41.073866+0000","last_became_peered":"2024-04-26T21:09:41.073866+0000","last_unstale":"2024-04-26T21:10:03.719794+0000","last_undegraded":"2024-04-26T21:10:03.719794+0000","last_fullsized":"2024-04-26T21:10:03.719794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:16.786155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692482+0000","last_change":"2024-04-26T21:09:41.074934+0000","last_active":"2024-04-26T21:10:03.692482+0000","last_peered":"2024-04-26T21:10:03.692482+0000","last_clean":"2024-04-26T21:10:03.692482+0000","last_became_active":"2024-04-26T21:09:41.072388+0000","last_became_peered":"2024-04-26T21:09:41.072388+0000","last_unstale":"2024-04-26T21:10:03.692482+0000","last_undegraded":"2024-04-26T21:10:03.692482+0000","last_fullsized":"2024-04-26T21:10:03.692482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:23.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716459+0000","last_change":"2024-04-26T21:09:41.062598+0000","last_active":"2024-04-26T21:10:03.716459+0000","last_peered":"2024-04-26T21:10:03.716459+0000","last_clean":"2024-04-26T21:10:03.716459+0000","last_became_active":"2024-04-26T21:09:41.061828+0000","last_became_peered":"2024-04-26T21:09:41.061828+0000","last_unstale":"2024-04-26T21:10:03.716459+0000","last_undegraded":"2024-04-26T21:10:03.716459+0000","last_fullsized":"2024-04-26T21:10:03.716459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:52.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716269+0000","last_change":"2024-04-26T21:09:41.064874+0000","last_active":"2024-04-26T21:10:03.716269+0000","last_peered":"2024-04-26T21:10:03.716269+0000","last_clean":"2024-04-26T21:10:03.716269+0000","last_became_active":"2024-04-26T21:09:41.060190+0000","last_became_peered":"2024-04-26T21:09:41.060190+0000","last_unstale":"2024-04-26T21:10:03.716269+0000","last_undegraded":"2024-04-26T21:10:03.716269+0000","last_fullsized":"2024-04-26T21:10:03.716269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697003+0000","last_change":"2024-04-26T21:09:41.067343+0000","last_active":"2024-04-26T21:10:03.697003+0000","last_peered":"2024-04-26T21:10:03.697003+0000","last_clean":"2024-04-26T21:10:03.697003+0000","last_became_active":"2024-04-26T21:09:41.066248+0000","last_became_peered":"2024-04-26T21:09:41.066248+0000","last_unstale":"2024-04-26T21:10:03.697003+0000","last_undegraded":"2024-04-26T21:10:03.697003+0000","last_fullsized":"2024-04-26T21:10:03.697003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:40.476089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697120+0000","last_change":"2024-04-26T21:09:41.079414+0000","last_active":"2024-04-26T21:10:03.697120+0000","last_peered":"2024-04-26T21:10:03.697120+0000","last_clean":"2024-04-26T21:10:03.697120+0000","last_became_active":"2024-04-26T21:09:41.078949+0000","last_became_peered":"2024-04-26T21:09:41.078949+0000","last_unstale":"2024-04-26T21:10:03.697120+0000","last_undegraded":"2024-04-26T21:10:03.697120+0000","last_fullsized":"2024-04-26T21:10:03.697120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:19.947290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697532+0000","last_change":"2024-04-26T21:09:41.069376+0000","last_active":"2024-04-26T21:10:03.697532+0000","last_peered":"2024-04-26T21:10:03.697532+0000","last_clean":"2024-04-26T21:10:03.697532+0000","last_became_active":"2024-04-26T21:09:41.069049+0000","last_became_peered":"2024-04-26T21:09:41.069049+0000","last_unstale":"2024-04-26T21:10:03.697532+0000","last_undegraded":"2024-04-26T21:10:03.697532+0000","last_fullsized":"2024-04-26T21:10:03.697532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:25.058292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.704488+0000","last_change":"2024-04-26T21:09:41.072485+0000","last_active":"2024-04-26T21:10:03.704488+0000","last_peered":"2024-04-26T21:10:03.704488+0000","last_clean":"2024-04-26T21:10:03.704488+0000","last_became_active":"2024-04-26T21:09:41.070990+0000","last_became_peered":"2024-04-26T21:09:41.070990+0000","last_unstale":"2024-04-26T21:10:03.704488+0000","last_undegraded":"2024-04-26T21:10:03.704488+0000","last_fullsized":"2024-04-26T21:10:03.704488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:40.648097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:07.883219+0000","last_change":"2024-04-26T21:10:03.739254+0000","last_active":"2024-04-26T21:11:07.883219+0000","last_peered":"2024-04-26T21:11:07.883219+0000","last_clean":"2024-04-26T21:11:07.883219+0000","last_became_active":"2024-04-26T21:10:03.709954+0000","last_became_peered":"2024-04-26T21:10:03.709954+0000","last_unstale":"2024-04-26T21:11:07.883219+0000","last_undegraded":"2024-04-26T21:11:07.883219+0000","last_fullsized":"2024-04-26T21:11:07.883219+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:33.358811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698762+0000","last_change":"2024-04-26T21:09:41.084249+0000","last_active":"2024-04-26T21:10:03.698762+0000","last_peered":"2024-04-26T21:10:03.698762+0000","last_clean":"2024-04-26T21:10:03.698762+0000","last_became_active":"2024-04-26T21:09:41.083580+0000","last_became_peered":"2024-04-26T21:09:41.083580+0000","last_unstale":"2024-04-26T21:10:03.698762+0000","last_undegraded":"2024-04-26T21:10:03.698762+0000","last_fullsized":"2024-04-26T21:10:03.698762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:02.615403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699085+0000","last_change":"2024-04-26T21:09:40.389376+0000","last_active":"2024-04-26T21:10:03.699085+0000","last_peered":"2024-04-26T21:10:03.699085+0000","last_clean":"2024-04-26T21:10:03.699085+0000","last_became_active":"2024-04-26T21:09:40.388761+0000","last_became_peered":"2024-04-26T21:09:40.388761+0000","last_unstale":"2024-04-26T21:10:03.699085+0000","last_undegraded":"2024-04-26T21:10:03.699085+0000","last_fullsized":"2024-04-26T21:10:03.699085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:39.940524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716712+0000","last_change":"2024-04-26T21:09:41.073193+0000","last_active":"2024-04-26T21:10:03.716712+0000","last_peered":"2024-04-26T21:10:03.716712+0000","last_clean":"2024-04-26T21:10:03.716712+0000","last_became_active":"2024-04-26T21:09:41.071569+0000","last_became_peered":"2024-04-26T21:09:41.071569+0000","last_unstale":"2024-04-26T21:10:03.716712+0000","last_undegraded":"2024-04-26T21:10:03.716712+0000","last_fullsized":"2024-04-26T21:10:03.716712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:40.078386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.708560+0000","last_change":"2024-04-26T21:09:40.394188+0000","last_active":"2024-04-26T21:10:03.708560+0000","last_peered":"2024-04-26T21:10:03.708560+0000","last_clean":"2024-04-26T21:10:03.708560+0000","last_became_active":"2024-04-26T21:09:40.393854+0000","last_became_peered":"2024-04-26T21:09:40.393854+0000","last_unstale":"2024-04-26T21:10:03.708560+0000","last_undegraded":"2024-04-26T21:10:03.708560+0000","last_fullsized":"2024-04-26T21:10:03.708560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:30.232353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697833+0000","last_change":"2024-04-26T21:09:41.068545+0000","last_active":"2024-04-26T21:10:03.697833+0000","last_peered":"2024-04-26T21:10:03.697833+0000","last_clean":"2024-04-26T21:10:03.697833+0000","last_became_active":"2024-04-26T21:09:41.066805+0000","last_became_peered":"2024-04-26T21:09:41.066805+0000","last_unstale":"2024-04-26T21:10:03.697833+0000","last_undegraded":"2024-04-26T21:10:03.697833+0000","last_fullsized":"2024-04-26T21:10:03.697833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:32.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.709517+0000","last_change":"2024-04-26T21:09:40.382727+0000","last_active":"2024-04-26T21:10:03.709517+0000","last_peered":"2024-04-26T21:10:03.709517+0000","last_clean":"2024-04-26T21:10:03.709517+0000","last_became_active":"2024-04-26T21:09:40.380205+0000","last_became_peered":"2024-04-26T21:09:40.380205+0000","last_unstale":"2024-04-26T21:10:03.709517+0000","last_undegraded":"2024-04-26T21:10:03.709517+0000","last_fullsized":"2024-04-26T21:10:03.709517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:32.981478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.722050+0000","last_change":"2024-04-26T21:10:03.721385+0000","last_active":"2024-04-26T21:10:03.722050+0000","last_peered":"2024-04-26T21:10:03.722050+0000","last_clean":"2024-04-26T21:10:03.722050+0000","last_became_active":"2024-04-26T21:10:03.721009+0000","last_became_peered":"2024-04-26T21:10:03.721009+0000","last_unstale":"2024-04-26T21:10:03.722050+0000","last_undegraded":"2024-04-26T21:10:03.722050+0000","last_fullsized":"2024-04-26T21:10:03.722050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:25.240987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":108,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:08.724922+0000","last_change":"2024-04-26T21:09:40.395282+0000","last_active":"2024-04-26T21:11:08.724922+0000","last_peered":"2024-04-26T21:11:08.724922+0000","last_clean":"2024-04-26T21:11:08.724922+0000","last_became_active":"2024-04-26T21:09:40.394858+0000","last_became_peered":"2024-04-26T21:09:40.394858+0000","last_unstale":"2024-04-26T21:11:08.724922+0000","last_undegraded":"2024-04-26T21:11:08.724922+0000","last_fullsized":"2024-04-26T21:11:08.724922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:38.330771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.709417+0000","last_change":"2024-04-26T21:09:41.064853+0000","last_active":"2024-04-26T21:10:03.709417+0000","last_peered":"2024-04-26T21:10:03.709417+0000","last_clean":"2024-04-26T21:10:03.709417+0000","last_became_active":"2024-04-26T21:09:41.064236+0000","last_became_peered":"2024-04-26T21:09:41.064236+0000","last_unstale":"2024-04-26T21:10:03.709417+0000","last_undegraded":"2024-04-26T21:10:03.709417+0000","last_fullsized":"2024-04-26T21:10:03.709417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:05.549027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700407+0000","last_change":"2024-04-26T21:09:40.390857+0000","last_active":"2024-04-26T21:10:03.700407+0000","last_peered":"2024-04-26T21:10:03.700407+0000","last_clean":"2024-04-26T21:10:03.700407+0000","last_became_active":"2024-04-26T21:09:40.390513+0000","last_became_peered":"2024-04-26T21:09:40.390513+0000","last_unstale":"2024-04-26T21:10:03.700407+0000","last_undegraded":"2024-04-26T21:10:03.700407+0000","last_fullsized":"2024-04-26T21:10:03.700407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700524+0000","last_change":"2024-04-26T21:10:03.700524+0000","last_active":"2024-04-26T21:10:03.700524+0000","last_peered":"2024-04-26T21:10:03.700524+0000","last_clean":"2024-04-26T21:10:03.700524+0000","last_became_active":"2024-04-26T21:10:03.699371+0000","last_became_peered":"2024-04-26T21:10:03.699371+0000","last_unstale":"2024-04-26T21:10:03.700524+0000","last_undegraded":"2024-04-26T21:10:03.700524+0000","last_fullsized":"2024-04-26T21:10:03.700524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:22.223665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691275+0000","last_change":"2024-04-26T21:09:40.390893+0000","last_active":"2024-04-26T21:10:03.691275+0000","last_peered":"2024-04-26T21:10:03.691275+0000","last_clean":"2024-04-26T21:10:03.691275+0000","last_became_active":"2024-04-26T21:09:40.390519+0000","last_became_peered":"2024-04-26T21:09:40.390519+0000","last_unstale":"2024-04-26T21:10:03.691275+0000","last_undegraded":"2024-04-26T21:10:03.691275+0000","last_fullsized":"2024-04-26T21:10:03.691275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:40.963614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698119+0000","last_change":"2024-04-26T21:09:41.074751+0000","last_active":"2024-04-26T21:10:03.698119+0000","last_peered":"2024-04-26T21:10:03.698119+0000","last_clean":"2024-04-26T21:10:03.698119+0000","last_became_active":"2024-04-26T21:09:41.074330+0000","last_became_peered":"2024-04-26T21:09:41.074330+0000","last_unstale":"2024-04-26T21:10:03.698119+0000","last_undegraded":"2024-04-26T21:10:03.698119+0000","last_fullsized":"2024-04-26T21:10:03.698119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:47.759543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690977+0000","last_change":"2024-04-26T21:09:40.389971+0000","last_active":"2024-04-26T21:10:03.690977+0000","last_peered":"2024-04-26T21:10:03.690977+0000","last_clean":"2024-04-26T21:10:03.690977+0000","last_became_active":"2024-04-26T21:09:40.389093+0000","last_became_peered":"2024-04-26T21:09:40.389093+0000","last_unstale":"2024-04-26T21:10:03.690977+0000","last_undegraded":"2024-04-26T21:10:03.690977+0000","last_fullsized":"2024-04-26T21:10:03.690977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:18.838994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698551+0000","last_change":"2024-04-26T21:09:41.068903+0000","last_active":"2024-04-26T21:10:03.698551+0000","last_peered":"2024-04-26T21:10:03.698551+0000","last_clean":"2024-04-26T21:10:03.698551+0000","last_became_active":"2024-04-26T21:09:41.068585+0000","last_became_peered":"2024-04-26T21:09:41.068585+0000","last_unstale":"2024-04-26T21:10:03.698551+0000","last_undegraded":"2024-04-26T21:10:03.698551+0000","last_fullsized":"2024-04-26T21:10:03.698551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:18.372213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":189,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.729225+0000","last_change":"2024-04-26T21:09:40.391501+0000","last_active":"2024-04-26T21:11:10.729225+0000","last_peered":"2024-04-26T21:11:10.729225+0000","last_clean":"2024-04-26T21:11:10.729225+0000","last_became_active":"2024-04-26T21:09:40.390434+0000","last_became_peered":"2024-04-26T21:09:40.390434+0000","last_unstale":"2024-04-26T21:11:10.729225+0000","last_undegraded":"2024-04-26T21:11:10.729225+0000","last_fullsized":"2024-04-26T21:11:10.729225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:11.854716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698702+0000","last_change":"2024-04-26T21:09:41.064937+0000","last_active":"2024-04-26T21:10:03.698702+0000","last_peered":"2024-04-26T21:10:03.698702+0000","last_clean":"2024-04-26T21:10:03.698702+0000","last_became_active":"2024-04-26T21:09:41.063667+0000","last_became_peered":"2024-04-26T21:09:41.063667+0000","last_unstale":"2024-04-26T21:10:03.698702+0000","last_undegraded":"2024-04-26T21:10:03.698702+0000","last_fullsized":"2024-04-26T21:10:03.698702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.380869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.765337+0000","last_change":"2024-04-26T21:10:03.765337+0000","last_active":"2024-04-26T21:10:03.765337+0000","last_peered":"2024-04-26T21:10:03.765337+0000","last_clean":"2024-04-26T21:10:03.765337+0000","last_became_active":"2024-04-26T21:10:03.722557+0000","last_became_peered":"2024-04-26T21:10:03.722557+0000","last_unstale":"2024-04-26T21:10:03.765337+0000","last_undegraded":"2024-04-26T21:10:03.765337+0000","last_fullsized":"2024-04-26T21:10:03.765337+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:34.370015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716620+0000","last_change":"2024-04-26T21:09:41.064559+0000","last_active":"2024-04-26T21:10:03.716620+0000","last_peered":"2024-04-26T21:10:03.716620+0000","last_clean":"2024-04-26T21:10:03.716620+0000","last_became_active":"2024-04-26T21:09:41.063117+0000","last_became_peered":"2024-04-26T21:09:41.063117+0000","last_unstale":"2024-04-26T21:10:03.716620+0000","last_undegraded":"2024-04-26T21:10:03.716620+0000","last_fullsized":"2024-04-26T21:10:03.716620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:34.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692110+0000","last_change":"2024-04-26T21:09:40.390507+0000","last_active":"2024-04-26T21:10:03.692110+0000","last_peered":"2024-04-26T21:10:03.692110+0000","last_clean":"2024-04-26T21:10:03.692110+0000","last_became_active":"2024-04-26T21:09:40.388503+0000","last_became_peered":"2024-04-26T21:09:40.388503+0000","last_unstale":"2024-04-26T21:10:03.692110+0000","last_undegraded":"2024-04-26T21:10:03.692110+0000","last_fullsized":"2024-04-26T21:10:03.692110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:40.432667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697601+0000","last_change":"2024-04-26T21:09:41.065188+0000","last_active":"2024-04-26T21:10:03.697601+0000","last_peered":"2024-04-26T21:10:03.697601+0000","last_clean":"2024-04-26T21:10:03.697601+0000","last_became_active":"2024-04-26T21:09:41.061218+0000","last_became_peered":"2024-04-26T21:09:41.061218+0000","last_unstale":"2024-04-26T21:10:03.697601+0000","last_undegraded":"2024-04-26T21:10:03.697601+0000","last_fullsized":"2024-04-26T21:10:03.697601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:42.087561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":242,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.927758+0000","last_change":"2024-04-26T21:10:03.756099+0000","last_active":"2024-04-26T21:11:10.927758+0000","last_peered":"2024-04-26T21:11:10.927758+0000","last_clean":"2024-04-26T21:11:10.927758+0000","last_became_active":"2024-04-26T21:10:03.723285+0000","last_became_peered":"2024-04-26T21:10:03.723285+0000","last_unstale":"2024-04-26T21:11:10.927758+0000","last_undegraded":"2024-04-26T21:11:10.927758+0000","last_fullsized":"2024-04-26T21:11:10.927758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:19.825308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":128,"num_write":31,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":48166,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707938+0000","last_change":"2024-04-26T21:09:41.063711+0000","last_active":"2024-04-26T21:10:03.707938+0000","last_peered":"2024-04-26T21:10:03.707938+0000","last_clean":"2024-04-26T21:10:03.707938+0000","last_became_active":"2024-04-26T21:09:41.063330+0000","last_became_peered":"2024-04-26T21:09:41.063330+0000","last_unstale":"2024-04-26T21:10:03.707938+0000","last_undegraded":"2024-04-26T21:10:03.707938+0000","last_fullsized":"2024-04-26T21:10:03.707938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:28.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697890+0000","last_change":"2024-04-26T21:09:40.404407+0000","last_active":"2024-04-26T21:10:03.697890+0000","last_peered":"2024-04-26T21:10:03.697890+0000","last_clean":"2024-04-26T21:10:03.697890+0000","last_became_active":"2024-04-26T21:09:40.402751+0000","last_became_peered":"2024-04-26T21:09:40.402751+0000","last_unstale":"2024-04-26T21:10:03.697890+0000","last_undegraded":"2024-04-26T21:10:03.697890+0000","last_fullsized":"2024-04-26T21:10:03.697890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:32.171086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691862+0000","last_change":"2024-04-26T21:09:41.079310+0000","last_active":"2024-04-26T21:10:03.691862+0000","last_peered":"2024-04-26T21:10:03.691862+0000","last_clean":"2024-04-26T21:10:03.691862+0000","last_became_active":"2024-04-26T21:09:41.077848+0000","last_became_peered":"2024-04-26T21:09:41.077848+0000","last_unstale":"2024-04-26T21:10:03.691862+0000","last_undegraded":"2024-04-26T21:10:03.691862+0000","last_fullsized":"2024-04-26T21:10:03.691862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:51.745349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691062+0000","last_change":"2024-04-26T21:09:40.391216+0000","last_active":"2024-04-26T21:10:03.691062+0000","last_peered":"2024-04-26T21:10:03.691062+0000","last_clean":"2024-04-26T21:10:03.691062+0000","last_became_active":"2024-04-26T21:09:40.390753+0000","last_became_peered":"2024-04-26T21:09:40.390753+0000","last_unstale":"2024-04-26T21:10:03.691062+0000","last_undegraded":"2024-04-26T21:10:03.691062+0000","last_fullsized":"2024-04-26T21:10:03.691062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:24.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690737+0000","last_change":"2024-04-26T21:09:41.079881+0000","last_active":"2024-04-26T21:10:03.690737+0000","last_peered":"2024-04-26T21:10:03.690737+0000","last_clean":"2024-04-26T21:10:03.690737+0000","last_became_active":"2024-04-26T21:09:41.079506+0000","last_became_peered":"2024-04-26T21:09:41.079506+0000","last_unstale":"2024-04-26T21:10:03.690737+0000","last_undegraded":"2024-04-26T21:10:03.690737+0000","last_fullsized":"2024-04-26T21:10:03.690737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:56.300294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":111,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.729231+0000","last_change":"2024-04-26T21:09:40.391458+0000","last_active":"2024-04-26T21:11:10.729231+0000","last_peered":"2024-04-26T21:11:10.729231+0000","last_clean":"2024-04-26T21:11:10.729231+0000","last_became_active":"2024-04-26T21:09:40.388494+0000","last_became_peered":"2024-04-26T21:09:40.388494+0000","last_unstale":"2024-04-26T21:11:10.729231+0000","last_undegraded":"2024-04-26T21:11:10.729231+0000","last_fullsized":"2024-04-26T21:11:10.729231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:07.523885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690197+0000","last_change":"2024-04-26T21:09:41.068278+0000","last_active":"2024-04-26T21:10:03.690197+0000","last_peered":"2024-04-26T21:10:03.690197+0000","last_clean":"2024-04-26T21:10:03.690197+0000","last_became_active":"2024-04-26T21:09:41.067865+0000","last_became_peered":"2024-04-26T21:09:41.067865+0000","last_unstale":"2024-04-26T21:10:03.690197+0000","last_undegraded":"2024-04-26T21:10:03.690197+0000","last_fullsized":"2024-04-26T21:10:03.690197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:19.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707555+0000","last_change":"2024-04-26T21:09:40.394390+0000","last_active":"2024-04-26T21:10:03.707555+0000","last_peered":"2024-04-26T21:10:03.707555+0000","last_clean":"2024-04-26T21:10:03.707555+0000","last_became_active":"2024-04-26T21:09:40.389728+0000","last_became_peered":"2024-04-26T21:09:40.389728+0000","last_unstale":"2024-04-26T21:10:03.707555+0000","last_undegraded":"2024-04-26T21:10:03.707555+0000","last_fullsized":"2024-04-26T21:10:03.707555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:42.301974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690944+0000","last_change":"2024-04-26T21:09:41.075703+0000","last_active":"2024-04-26T21:10:03.690944+0000","last_peered":"2024-04-26T21:10:03.690944+0000","last_clean":"2024-04-26T21:10:03.690944+0000","last_became_active":"2024-04-26T21:09:41.075310+0000","last_became_peered":"2024-04-26T21:09:41.075310+0000","last_unstale":"2024-04-26T21:10:03.690944+0000","last_undegraded":"2024-04-26T21:10:03.690944+0000","last_fullsized":"2024-04-26T21:10:03.690944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:20.030941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.710094+0000","last_change":"2024-04-26T21:09:40.381293+0000","last_active":"2024-04-26T21:10:03.710094+0000","last_peered":"2024-04-26T21:10:03.710094+0000","last_clean":"2024-04-26T21:10:03.710094+0000","last_became_active":"2024-04-26T21:09:40.380287+0000","last_became_peered":"2024-04-26T21:09:40.380287+0000","last_unstale":"2024-04-26T21:10:03.710094+0000","last_undegraded":"2024-04-26T21:10:03.710094+0000","last_fullsized":"2024-04-26T21:10:03.710094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:47.479546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698460+0000","last_change":"2024-04-26T21:09:41.077928+0000","last_active":"2024-04-26T21:10:03.698460+0000","last_peered":"2024-04-26T21:10:03.698460+0000","last_clean":"2024-04-26T21:10:03.698460+0000","last_became_active":"2024-04-26T21:09:41.076611+0000","last_became_peered":"2024-04-26T21:09:41.076611+0000","last_unstale":"2024-04-26T21:10:03.698460+0000","last_undegraded":"2024-04-26T21:10:03.698460+0000","last_fullsized":"2024-04-26T21:10:03.698460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697596+0000","last_change":"2024-04-26T21:09:40.394161+0000","last_active":"2024-04-26T21:10:03.697596+0000","last_peered":"2024-04-26T21:10:03.697596+0000","last_clean":"2024-04-26T21:10:03.697596+0000","last_became_active":"2024-04-26T21:09:40.393277+0000","last_became_peered":"2024-04-26T21:09:40.393277+0000","last_unstale":"2024-04-26T21:10:03.697596+0000","last_undegraded":"2024-04-26T21:10:03.697596+0000","last_fullsized":"2024-04-26T21:10:03.697596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:32.408080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701600+0000","last_change":"2024-04-26T21:09:41.070926+0000","last_active":"2024-04-26T21:10:03.701600+0000","last_peered":"2024-04-26T21:10:03.701600+0000","last_clean":"2024-04-26T21:10:03.701600+0000","last_became_active":"2024-04-26T21:09:41.070260+0000","last_became_peered":"2024-04-26T21:09:41.070260+0000","last_unstale":"2024-04-26T21:10:03.701600+0000","last_undegraded":"2024-04-26T21:10:03.701600+0000","last_fullsized":"2024-04-26T21:10:03.701600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:22.916960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:09.889969+0000","last_change":"2024-04-26T21:09:40.395088+0000","last_active":"2024-04-26T21:11:09.889969+0000","last_peered":"2024-04-26T21:11:09.889969+0000","last_clean":"2024-04-26T21:11:09.889969+0000","last_became_active":"2024-04-26T21:09:40.393921+0000","last_became_peered":"2024-04-26T21:09:40.393921+0000","last_unstale":"2024-04-26T21:11:09.889969+0000","last_undegraded":"2024-04-26T21:11:09.889969+0000","last_fullsized":"2024-04-26T21:11:09.889969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:30.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701596+0000","last_change":"2024-04-26T21:09:41.064834+0000","last_active":"2024-04-26T21:10:03.701596+0000","last_peered":"2024-04-26T21:10:03.701596+0000","last_clean":"2024-04-26T21:10:03.701596+0000","last_became_active":"2024-04-26T21:09:41.063875+0000","last_became_peered":"2024-04-26T21:09:41.063875+0000","last_unstale":"2024-04-26T21:10:03.701596+0000","last_undegraded":"2024-04-26T21:10:03.701596+0000","last_fullsized":"2024-04-26T21:10:03.701596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:48.438740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691737+0000","last_change":"2024-04-26T21:09:41.075181+0000","last_active":"2024-04-26T21:10:03.691737+0000","last_peered":"2024-04-26T21:10:03.691737+0000","last_clean":"2024-04-26T21:10:03.691737+0000","last_became_active":"2024-04-26T21:09:41.073137+0000","last_became_peered":"2024-04-26T21:09:41.073137+0000","last_unstale":"2024-04-26T21:10:03.691737+0000","last_undegraded":"2024-04-26T21:10:03.691737+0000","last_fullsized":"2024-04-26T21:10:03.691737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:08.609048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":105,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:06.716611+0000","last_change":"2024-04-26T21:09:40.404159+0000","last_active":"2024-04-26T21:11:06.716611+0000","last_peered":"2024-04-26T21:11:06.716611+0000","last_clean":"2024-04-26T21:11:06.716611+0000","last_became_active":"2024-04-26T21:09:40.403828+0000","last_became_peered":"2024-04-26T21:09:40.403828+0000","last_unstale":"2024-04-26T21:11:06.716611+0000","last_undegraded":"2024-04-26T21:11:06.716611+0000","last_fullsized":"2024-04-26T21:11:06.716611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:51.245405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698753+0000","last_change":"2024-04-26T21:09:41.068913+0000","last_active":"2024-04-26T21:10:03.698753+0000","last_peered":"2024-04-26T21:10:03.698753+0000","last_clean":"2024-04-26T21:10:03.698753+0000","last_became_active":"2024-04-26T21:09:41.068534+0000","last_became_peered":"2024-04-26T21:09:41.068534+0000","last_unstale":"2024-04-26T21:10:03.698753+0000","last_undegraded":"2024-04-26T21:10:03.698753+0000","last_fullsized":"2024-04-26T21:10:03.698753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:39.739853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719296+0000","last_change":"2024-04-26T21:09:40.387055+0000","last_active":"2024-04-26T21:10:03.719296+0000","last_peered":"2024-04-26T21:10:03.719296+0000","last_clean":"2024-04-26T21:10:03.719296+0000","last_became_active":"2024-04-26T21:09:40.386684+0000","last_became_peered":"2024-04-26T21:09:40.386684+0000","last_unstale":"2024-04-26T21:10:03.719296+0000","last_undegraded":"2024-04-26T21:10:03.719296+0000","last_fullsized":"2024-04-26T21:10:03.719296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:02.871494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690520+0000","last_change":"2024-04-26T21:08:05.429394+0000","last_active":"2024-04-26T21:10:03.690520+0000","last_peered":"2024-04-26T21:10:03.690520+0000","last_clean":"2024-04-26T21:10:03.690520+0000","last_became_active":"2024-04-26T21:08:05.428055+0000","last_became_peered":"2024-04-26T21:08:05.428055+0000","last_unstale":"2024-04-26T21:10:03.690520+0000","last_undegraded":"2024-04-26T21:10:03.690520+0000","last_fullsized":"2024-04-26T21:10:03.690520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_clean_scrub_stamp":"2024-04-26T21:03:22.595431+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:23.104153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691138+0000","last_change":"2024-04-26T21:09:41.080213+0000","last_active":"2024-04-26T21:10:03.691138+0000","last_peered":"2024-04-26T21:10:03.691138+0000","last_clean":"2024-04-26T21:10:03.691138+0000","last_became_active":"2024-04-26T21:09:41.076313+0000","last_became_peered":"2024-04-26T21:09:41.076313+0000","last_unstale":"2024-04-26T21:10:03.691138+0000","last_undegraded":"2024-04-26T21:10:03.691138+0000","last_fullsized":"2024-04-26T21:10:03.691138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:48.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691598+0000","last_change":"2024-04-26T21:09:40.387563+0000","last_active":"2024-04-26T21:10:03.691598+0000","last_peered":"2024-04-26T21:10:03.691598+0000","last_clean":"2024-04-26T21:10:03.691598+0000","last_became_active":"2024-04-26T21:09:40.383243+0000","last_became_peered":"2024-04-26T21:09:40.383243+0000","last_unstale":"2024-04-26T21:10:03.691598+0000","last_undegraded":"2024-04-26T21:10:03.691598+0000","last_fullsized":"2024-04-26T21:10:03.691598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:16.905074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.708245+0000","last_change":"2024-04-26T21:09:40.398078+0000","last_active":"2024-04-26T21:10:03.708245+0000","last_peered":"2024-04-26T21:10:03.708245+0000","last_clean":"2024-04-26T21:10:03.708245+0000","last_became_active":"2024-04-26T21:09:40.397007+0000","last_became_peered":"2024-04-26T21:09:40.397007+0000","last_unstale":"2024-04-26T21:10:03.708245+0000","last_undegraded":"2024-04-26T21:10:03.708245+0000","last_fullsized":"2024-04-26T21:10:03.708245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:18.374678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692204+0000","last_change":"2024-04-26T21:09:41.063430+0000","last_active":"2024-04-26T21:10:03.692204+0000","last_peered":"2024-04-26T21:10:03.692204+0000","last_clean":"2024-04-26T21:10:03.692204+0000","last_became_active":"2024-04-26T21:09:41.062591+0000","last_became_peered":"2024-04-26T21:09:41.062591+0000","last_unstale":"2024-04-26T21:10:03.692204+0000","last_undegraded":"2024-04-26T21:10:03.692204+0000","last_fullsized":"2024-04-26T21:10:03.692204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:58.874576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697875+0000","last_change":"2024-04-26T21:09:40.378465+0000","last_active":"2024-04-26T21:10:03.697875+0000","last_peered":"2024-04-26T21:10:03.697875+0000","last_clean":"2024-04-26T21:10:03.697875+0000","last_became_active":"2024-04-26T21:09:40.378089+0000","last_became_peered":"2024-04-26T21:09:40.378089+0000","last_unstale":"2024-04-26T21:10:03.697875+0000","last_undegraded":"2024-04-26T21:10:03.697875+0000","last_fullsized":"2024-04-26T21:10:03.697875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:56.425077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699449+0000","last_change":"2024-04-26T21:09:41.080446+0000","last_active":"2024-04-26T21:10:03.699449+0000","last_peered":"2024-04-26T21:10:03.699449+0000","last_clean":"2024-04-26T21:10:03.699449+0000","last_became_active":"2024-04-26T21:09:41.074363+0000","last_became_peered":"2024-04-26T21:09:41.074363+0000","last_unstale":"2024-04-26T21:10:03.699449+0000","last_undegraded":"2024-04-26T21:10:03.699449+0000","last_fullsized":"2024-04-26T21:10:03.699449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:14.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719194+0000","last_change":"2024-04-26T21:09:40.403106+0000","last_active":"2024-04-26T21:10:03.719194+0000","last_peered":"2024-04-26T21:10:03.719194+0000","last_clean":"2024-04-26T21:10:03.719194+0000","last_became_active":"2024-04-26T21:09:40.402630+0000","last_became_peered":"2024-04-26T21:09:40.402630+0000","last_unstale":"2024-04-26T21:10:03.719194+0000","last_undegraded":"2024-04-26T21:10:03.719194+0000","last_fullsized":"2024-04-26T21:10:03.719194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:37.555164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716417+0000","last_change":"2024-04-26T21:09:41.069407+0000","last_active":"2024-04-26T21:10:03.716417+0000","last_peered":"2024-04-26T21:10:03.716417+0000","last_clean":"2024-04-26T21:10:03.716417+0000","last_became_active":"2024-04-26T21:09:41.067522+0000","last_became_peered":"2024-04-26T21:09:41.067522+0000","last_unstale":"2024-04-26T21:10:03.716417+0000","last_undegraded":"2024-04-26T21:10:03.716417+0000","last_fullsized":"2024-04-26T21:10:03.716417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.338665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.696784+0000","last_change":"2024-04-26T21:09:40.391323+0000","last_active":"2024-04-26T21:10:03.696784+0000","last_peered":"2024-04-26T21:10:03.696784+0000","last_clean":"2024-04-26T21:10:03.696784+0000","last_became_active":"2024-04-26T21:09:40.390982+0000","last_became_peered":"2024-04-26T21:09:40.390982+0000","last_unstale":"2024-04-26T21:10:03.696784+0000","last_undegraded":"2024-04-26T21:10:03.696784+0000","last_fullsized":"2024-04-26T21:10:03.696784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:03.465013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707438+0000","last_change":"2024-04-26T21:09:41.068305+0000","last_active":"2024-04-26T21:10:03.707438+0000","last_peered":"2024-04-26T21:10:03.707438+0000","last_clean":"2024-04-26T21:10:03.707438+0000","last_became_active":"2024-04-26T21:09:41.067946+0000","last_became_peered":"2024-04-26T21:09:41.067946+0000","last_unstale":"2024-04-26T21:10:03.707438+0000","last_undegraded":"2024-04-26T21:10:03.707438+0000","last_fullsized":"2024-04-26T21:10:03.707438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:38.761142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698273+0000","last_change":"2024-04-26T21:09:40.387004+0000","last_active":"2024-04-26T21:10:03.698273+0000","last_peered":"2024-04-26T21:10:03.698273+0000","last_clean":"2024-04-26T21:10:03.698273+0000","last_became_active":"2024-04-26T21:09:40.382385+0000","last_became_peered":"2024-04-26T21:09:40.382385+0000","last_unstale":"2024-04-26T21:10:03.698273+0000","last_undegraded":"2024-04-26T21:10:03.698273+0000","last_fullsized":"2024-04-26T21:10:03.698273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:40.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.696898+0000","last_change":"2024-04-26T21:09:41.071024+0000","last_active":"2024-04-26T21:10:03.696898+0000","last_peered":"2024-04-26T21:10:03.696898+0000","last_clean":"2024-04-26T21:10:03.696898+0000","last_became_active":"2024-04-26T21:09:41.070456+0000","last_became_peered":"2024-04-26T21:09:41.070456+0000","last_unstale":"2024-04-26T21:10:03.696898+0000","last_undegraded":"2024-04-26T21:10:03.696898+0000","last_fullsized":"2024-04-26T21:10:03.696898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:56.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701415+0000","last_change":"2024-04-26T21:09:40.404128+0000","last_active":"2024-04-26T21:10:03.701415+0000","last_peered":"2024-04-26T21:10:03.701415+0000","last_clean":"2024-04-26T21:10:03.701415+0000","last_became_active":"2024-04-26T21:09:40.403760+0000","last_became_peered":"2024-04-26T21:09:40.403760+0000","last_unstale":"2024-04-26T21:10:03.701415+0000","last_undegraded":"2024-04-26T21:10:03.701415+0000","last_fullsized":"2024-04-26T21:10:03.701415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698992+0000","last_change":"2024-04-26T21:09:41.067676+0000","last_active":"2024-04-26T21:10:03.698992+0000","last_peered":"2024-04-26T21:10:03.698992+0000","last_clean":"2024-04-26T21:10:03.698992+0000","last_became_active":"2024-04-26T21:09:41.066736+0000","last_became_peered":"2024-04-26T21:09:41.066736+0000","last_unstale":"2024-04-26T21:10:03.698992+0000","last_undegraded":"2024-04-26T21:10:03.698992+0000","last_fullsized":"2024-04-26T21:10:03.698992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:39.477690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.724113+0000","last_change":"2024-04-26T21:10:03.722877+0000","last_active":"2024-04-26T21:10:03.724113+0000","last_peered":"2024-04-26T21:10:03.724113+0000","last_clean":"2024-04-26T21:10:03.724113+0000","last_became_active":"2024-04-26T21:10:03.710433+0000","last_became_peered":"2024-04-26T21:10:03.710433+0000","last_unstale":"2024-04-26T21:10:03.724113+0000","last_undegraded":"2024-04-26T21:10:03.724113+0000","last_fullsized":"2024-04-26T21:10:03.724113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:59.042373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691088+0000","last_change":"2024-04-26T21:09:41.074270+0000","last_active":"2024-04-26T21:10:03.691088+0000","last_peered":"2024-04-26T21:10:03.691088+0000","last_clean":"2024-04-26T21:10:03.691088+0000","last_became_active":"2024-04-26T21:09:41.073847+0000","last_became_peered":"2024-04-26T21:09:41.073847+0000","last_unstale":"2024-04-26T21:10:03.691088+0000","last_undegraded":"2024-04-26T21:10:03.691088+0000","last_fullsized":"2024-04-26T21:10:03.691088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:03.737883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":105,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:05.715799+0000","last_change":"2024-04-26T21:09:40.397644+0000","last_active":"2024-04-26T21:11:05.715799+0000","last_peered":"2024-04-26T21:11:05.715799+0000","last_clean":"2024-04-26T21:11:05.715799+0000","last_became_active":"2024-04-26T21:09:40.397234+0000","last_became_peered":"2024-04-26T21:09:40.397234+0000","last_unstale":"2024-04-26T21:11:05.715799+0000","last_undegraded":"2024-04-26T21:11:05.715799+0000","last_fullsized":"2024-04-26T21:11:05.715799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:32.220330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691262+0000","last_change":"2024-04-26T21:09:41.072171+0000","last_active":"2024-04-26T21:10:03.691262+0000","last_peered":"2024-04-26T21:10:03.691262+0000","last_clean":"2024-04-26T21:10:03.691262+0000","last_became_active":"2024-04-26T21:09:41.071706+0000","last_became_peered":"2024-04-26T21:09:41.071706+0000","last_unstale":"2024-04-26T21:10:03.691262+0000","last_undegraded":"2024-04-26T21:10:03.691262+0000","last_fullsized":"2024-04-26T21:10:03.691262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:14.286865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691309+0000","last_change":"2024-04-26T21:09:40.393724+0000","last_active":"2024-04-26T21:10:03.691309+0000","last_peered":"2024-04-26T21:10:03.691309+0000","last_clean":"2024-04-26T21:10:03.691309+0000","last_became_active":"2024-04-26T21:09:40.392441+0000","last_became_peered":"2024-04-26T21:09:40.392441+0000","last_unstale":"2024-04-26T21:10:03.691309+0000","last_undegraded":"2024-04-26T21:10:03.691309+0000","last_fullsized":"2024-04-26T21:10:03.691309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:11.989418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.717632+0000","last_change":"2024-04-26T21:10:03.717632+0000","last_active":"2024-04-26T21:10:03.717632+0000","last_peered":"2024-04-26T21:10:03.717632+0000","last_clean":"2024-04-26T21:10:03.717632+0000","last_became_active":"2024-04-26T21:10:03.705478+0000","last_became_peered":"2024-04-26T21:10:03.705478+0000","last_unstale":"2024-04-26T21:10:03.717632+0000","last_undegraded":"2024-04-26T21:10:03.717632+0000","last_fullsized":"2024-04-26T21:10:03.717632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.259935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691920+0000","last_change":"2024-04-26T21:09:40.400884+0000","last_active":"2024-04-26T21:10:03.691920+0000","last_peered":"2024-04-26T21:10:03.691920+0000","last_clean":"2024-04-26T21:10:03.691920+0000","last_became_active":"2024-04-26T21:09:40.400326+0000","last_became_peered":"2024-04-26T21:09:40.400326+0000","last_unstale":"2024-04-26T21:10:03.691920+0000","last_undegraded":"2024-04-26T21:10:03.691920+0000","last_fullsized":"2024-04-26T21:10:03.691920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:13.114233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691193+0000","last_change":"2024-04-26T21:09:41.071959+0000","last_active":"2024-04-26T21:10:03.691193+0000","last_peered":"2024-04-26T21:10:03.691193+0000","last_clean":"2024-04-26T21:10:03.691193+0000","last_became_active":"2024-04-26T21:09:41.071601+0000","last_became_peered":"2024-04-26T21:09:41.071601+0000","last_unstale":"2024-04-26T21:10:03.691193+0000","last_undegraded":"2024-04-26T21:10:03.691193+0000","last_fullsized":"2024-04-26T21:10:03.691193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:29.806956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697259+0000","last_change":"2024-04-26T21:09:41.066764+0000","last_active":"2024-04-26T21:10:03.697259+0000","last_peered":"2024-04-26T21:10:03.697259+0000","last_clean":"2024-04-26T21:10:03.697259+0000","last_became_active":"2024-04-26T21:09:41.066333+0000","last_became_peered":"2024-04-26T21:09:41.066333+0000","last_unstale":"2024-04-26T21:10:03.697259+0000","last_undegraded":"2024-04-26T21:10:03.697259+0000","last_fullsized":"2024-04-26T21:10:03.697259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:32.680215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700693+0000","last_change":"2024-04-26T21:09:41.072567+0000","last_active":"2024-04-26T21:10:03.700693+0000","last_peered":"2024-04-26T21:10:03.700693+0000","last_clean":"2024-04-26T21:10:03.700693+0000","last_became_active":"2024-04-26T21:09:41.072033+0000","last_became_peered":"2024-04-26T21:09:41.072033+0000","last_unstale":"2024-04-26T21:10:03.700693+0000","last_undegraded":"2024-04-26T21:10:03.700693+0000","last_fullsized":"2024-04-26T21:10:03.700693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:37.178970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701116+0000","last_change":"2024-04-26T21:09:41.071111+0000","last_active":"2024-04-26T21:10:03.701116+0000","last_peered":"2024-04-26T21:10:03.701116+0000","last_clean":"2024-04-26T21:10:03.701116+0000","last_became_active":"2024-04-26T21:09:41.070354+0000","last_became_peered":"2024-04-26T21:09:41.070354+0000","last_unstale":"2024-04-26T21:10:03.701116+0000","last_undegraded":"2024-04-26T21:10:03.701116+0000","last_fullsized":"2024-04-26T21:10:03.701116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:10.990058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716445+0000","last_change":"2024-04-26T21:09:41.076927+0000","last_active":"2024-04-26T21:10:03.716445+0000","last_peered":"2024-04-26T21:10:03.716445+0000","last_clean":"2024-04-26T21:10:03.716445+0000","last_became_active":"2024-04-26T21:09:41.076600+0000","last_became_peered":"2024-04-26T21:09:41.076600+0000","last_unstale":"2024-04-26T21:10:03.716445+0000","last_undegraded":"2024-04-26T21:10:03.716445+0000","last_fullsized":"2024-04-26T21:10:03.716445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:25.960409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691916+0000","last_change":"2024-04-26T21:09:41.079293+0000","last_active":"2024-04-26T21:10:03.691916+0000","last_peered":"2024-04-26T21:10:03.691916+0000","last_clean":"2024-04-26T21:10:03.691916+0000","last_became_active":"2024-04-26T21:09:41.077473+0000","last_became_peered":"2024-04-26T21:09:41.077473+0000","last_unstale":"2024-04-26T21:10:03.691916+0000","last_undegraded":"2024-04-26T21:10:03.691916+0000","last_fullsized":"2024-04-26T21:10:03.691916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:47.281535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697787+0000","last_change":"2024-04-26T21:09:41.065914+0000","last_active":"2024-04-26T21:10:03.697787+0000","last_peered":"2024-04-26T21:10:03.697787+0000","last_clean":"2024-04-26T21:10:03.697787+0000","last_became_active":"2024-04-26T21:09:41.065477+0000","last_became_peered":"2024-04-26T21:09:41.065477+0000","last_unstale":"2024-04-26T21:10:03.697787+0000","last_undegraded":"2024-04-26T21:10:03.697787+0000","last_fullsized":"2024-04-26T21:10:03.697787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:40.517174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.704127+0000","last_change":"2024-04-26T21:09:41.068594+0000","last_active":"2024-04-26T21:10:03.704127+0000","last_peered":"2024-04-26T21:10:03.704127+0000","last_clean":"2024-04-26T21:10:03.704127+0000","last_became_active":"2024-04-26T21:09:41.068238+0000","last_became_peered":"2024-04-26T21:09:41.068238+0000","last_unstale":"2024-04-26T21:10:03.704127+0000","last_undegraded":"2024-04-26T21:10:03.704127+0000","last_fullsized":"2024-04-26T21:10:03.704127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:58.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699267+0000","last_change":"2024-04-26T21:09:41.081684+0000","last_active":"2024-04-26T21:10:03.699267+0000","last_peered":"2024-04-26T21:10:03.699267+0000","last_clean":"2024-04-26T21:10:03.699267+0000","last_became_active":"2024-04-26T21:09:41.074963+0000","last_became_peered":"2024-04-26T21:09:41.074963+0000","last_unstale":"2024-04-26T21:10:03.699267+0000","last_undegraded":"2024-04-26T21:10:03.699267+0000","last_fullsized":"2024-04-26T21:10:03.699267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:14.736390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691381+0000","last_change":"2024-04-26T21:09:41.068052+0000","last_active":"2024-04-26T21:10:03.691381+0000","last_peered":"2024-04-26T21:10:03.691381+0000","last_clean":"2024-04-26T21:10:03.691381+0000","last_became_active":"2024-04-26T21:09:41.067728+0000","last_became_peered":"2024-04-26T21:09:41.067728+0000","last_unstale":"2024-04-26T21:10:03.691381+0000","last_undegraded":"2024-04-26T21:10:03.691381+0000","last_fullsized":"2024-04-26T21:10:03.691381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:19.186963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698395+0000","last_change":"2024-04-26T21:09:41.078725+0000","last_active":"2024-04-26T21:10:03.698395+0000","last_peered":"2024-04-26T21:10:03.698395+0000","last_clean":"2024-04-26T21:10:03.698395+0000","last_became_active":"2024-04-26T21:09:41.075757+0000","last_became_peered":"2024-04-26T21:09:41.075757+0000","last_unstale":"2024-04-26T21:10:03.698395+0000","last_undegraded":"2024-04-26T21:10:03.698395+0000","last_fullsized":"2024-04-26T21:10:03.698395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:50.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697276+0000","last_change":"2024-04-26T21:09:41.067872+0000","last_active":"2024-04-26T21:10:03.697276+0000","last_peered":"2024-04-26T21:10:03.697276+0000","last_clean":"2024-04-26T21:10:03.697276+0000","last_became_active":"2024-04-26T21:09:41.067476+0000","last_became_peered":"2024-04-26T21:09:41.067476+0000","last_unstale":"2024-04-26T21:10:03.697276+0000","last_undegraded":"2024-04-26T21:10:03.697276+0000","last_fullsized":"2024-04-26T21:10:03.697276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:54.740719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692241+0000","last_change":"2024-04-26T21:09:41.080571+0000","last_active":"2024-04-26T21:10:03.692241+0000","last_peered":"2024-04-26T21:10:03.692241+0000","last_clean":"2024-04-26T21:10:03.692241+0000","last_became_active":"2024-04-26T21:09:41.078465+0000","last_became_peered":"2024-04-26T21:09:41.078465+0000","last_unstale":"2024-04-26T21:10:03.692241+0000","last_undegraded":"2024-04-26T21:10:03.692241+0000","last_fullsized":"2024-04-26T21:10:03.692241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:27.636693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697764+0000","last_change":"2024-04-26T21:09:41.080111+0000","last_active":"2024-04-26T21:10:03.697764+0000","last_peered":"2024-04-26T21:10:03.697764+0000","last_clean":"2024-04-26T21:10:03.697764+0000","last_became_active":"2024-04-26T21:09:41.079220+0000","last_became_peered":"2024-04-26T21:09:41.079220+0000","last_unstale":"2024-04-26T21:10:03.697764+0000","last_undegraded":"2024-04-26T21:10:03.697764+0000","last_fullsized":"2024-04-26T21:10:03.697764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.055957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60195,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":833,"num_read_kb":475,"num_write":238,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":48838,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":180585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":471504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1115634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710767,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":470879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972408,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":470468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1062986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":470479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":525446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":471818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":471612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":470187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":524113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1622016,"data_stored":1062945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":55806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:11:14.448 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:11:14.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:14 smithi027 ceph-mon[28085]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:15.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:14 smithi204 ceph-mon[32145]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:15.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:14 smithi094 ceph-mon[31273]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:15.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:15 smithi027 ceph-mon[28085]: pgmap v416: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:16.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:15 smithi204 ceph-mon[32145]: pgmap v416: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:16.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:15 smithi094 ceph-mon[31273]: pgmap v416: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 -- 172.21.15.27:0/194213808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955806bb50 msgr2=0x7f955806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 --2- 172.21.15.27:0/194213808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955806bb50 0x7f955806bf30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f954c009990 tx=0x7f954c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 -- 172.21.15.27:0/194213808 shutdown_connections 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 --2- 172.21.15.27:0/194213808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955810ff00 0x7f95581122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 --2- 172.21.15.27:0/194213808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955806bb50 0x7f955806bf30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 --2- 172.21.15.27:0/194213808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806c470 0x7f955810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 -- 172.21.15.27:0/194213808 >> 172.21.15.27:0/194213808 conn(0x7f95580792c0 msgr2=0x7f95580796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 -- 172.21.15.27:0/194213808 shutdown_connections 2024-04-26T21:11:17.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.277+0000 7f955fe00700 1 -- 172.21.15.27:0/194213808 wait complete. 2024-04-26T21:11:17.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.278+0000 7f955fe00700 1 Processor -- start 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 -- start start 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806bb50 0x7f955819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 0x7f955819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95581157c0 con 0x7f955810ff00 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9558115640 con 0x7f955806bb50 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955fe00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9558115940 con 0x7f955806c470 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.279+0000 7f955de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40722/0 (socket says 172.21.15.27:40722) 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 -- 172.21.15.27:0/312877926 learned_addr learned my addr 172.21.15.27:0/312877926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955d400700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806bb50 0x7f955819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:17.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955ca00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 0x7f955819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:17.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 -- 172.21.15.27:0/312877926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 msgr2=0x7f955819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 0x7f955819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 -- 172.21.15.27:0/312877926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806bb50 msgr2=0x7f955819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806bb50 0x7f955819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 -- 172.21.15.27:0/312877926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95581a0de0 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955ca00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 0x7f955819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.280+0000 7f955de00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f955000b9e0 tx=0x7f955000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.281+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550019070 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.281+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f955000cdc0 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.281+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95581a10d0 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.281+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550014e50 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.281+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95581a8950 con 0x7f955810ff00 2024-04-26T21:11:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.282+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9558102930 con 0x7f955810ff00 2024-04-26T21:11:17.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.284+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f955001c070 con 0x7f955810ff00 2024-04-26T21:11:17.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.286+0000 7f9556000700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 0x7f95400a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:17.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.286+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f95500cf7f0 con 0x7f955810ff00 2024-04-26T21:11:17.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.286+0000 7f955d400700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 0x7f95400a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:17.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.287+0000 7f955d400700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 0x7f95400a6330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b5c0 tx=0x7f954c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:17.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.290+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f955009c110 con 0x7f955810ff00 2024-04-26T21:11:17.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.516+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9558063860 con 0x7f95400a3e70 2024-04-26T21:11:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.526+0000 7f9556000700 1 -- 172.21.15.27:0/312877926 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258779 (secure 0 0 0) 0x7f9558063860 con 0x7f95400a3e70 2024-04-26T21:11:17.528 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 msgr2=0x7f95400a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 0x7f95400a6330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b5c0 tx=0x7f954c005fd0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 msgr2=0x7f95581a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f955000b9e0 tx=0x7f955000bcf0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 shutdown_connections 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f95400a3e70 0x7f95400a6330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955810ff00 0x7f95581a06a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955806bb50 0x7f955819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 --2- 172.21.15.27:0/312877926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955806c470 0x7f955819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 >> 172.21.15.27:0/312877926 conn(0x7f95580792c0 msgr2=0x7f9558111870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 shutdown_connections 2024-04-26T21:11:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:17.533+0000 7f955fe00700 1 -- 172.21.15.27:0/312877926 wait complete. 2024-04-26T21:11:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-26T21:11:17.940 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:17 smithi027 ceph-mon[28085]: pgmap v417: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:18.003 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:17 smithi204 ceph-mon[32145]: pgmap v417: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:18.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:17 smithi094 ceph-mon[31273]: pgmap v417: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:18.403 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-04-26T21:11:16.083806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650797,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":989,"num_read_kb":604,"num_write":421,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1229691,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396268,"kb_used_data":13764,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545556,"statfs":{"total":1151940427776,"available":1151534649344,"internally_reserved":0,"allocated":14094336,"data_stored":7588071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005362"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697923+0000","last_change":"2024-04-26T21:09:41.072073+0000","last_active":"2024-04-26T21:10:03.697923+0000","last_peered":"2024-04-26T21:10:03.697923+0000","last_clean":"2024-04-26T21:10:03.697923+0000","last_became_active":"2024-04-26T21:09:41.071687+0000","last_became_peered":"2024-04-26T21:09:41.071687+0000","last_unstale":"2024-04-26T21:10:03.697923+0000","last_undegraded":"2024-04-26T21:10:03.697923+0000","last_fullsized":"2024-04-26T21:10:03.697923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:17.092134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716245+0000","last_change":"2024-04-26T21:09:41.063568+0000","last_active":"2024-04-26T21:10:03.716245+0000","last_peered":"2024-04-26T21:10:03.716245+0000","last_clean":"2024-04-26T21:10:03.716245+0000","last_became_active":"2024-04-26T21:09:41.061712+0000","last_became_peered":"2024-04-26T21:09:41.061712+0000","last_unstale":"2024-04-26T21:10:03.716245+0000","last_undegraded":"2024-04-26T21:10:03.716245+0000","last_fullsized":"2024-04-26T21:10:03.716245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:38.894168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716760+0000","last_change":"2024-04-26T21:09:41.068164+0000","last_active":"2024-04-26T21:10:03.716760+0000","last_peered":"2024-04-26T21:10:03.716760+0000","last_clean":"2024-04-26T21:10:03.716760+0000","last_became_active":"2024-04-26T21:09:41.067074+0000","last_became_peered":"2024-04-26T21:09:41.067074+0000","last_unstale":"2024-04-26T21:10:03.716760+0000","last_undegraded":"2024-04-26T21:10:03.716760+0000","last_fullsized":"2024-04-26T21:10:03.716760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.900193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.714910+0000","last_change":"2024-04-26T21:10:03.713993+0000","last_active":"2024-04-26T21:10:03.714910+0000","last_peered":"2024-04-26T21:10:03.714910+0000","last_clean":"2024-04-26T21:10:03.714910+0000","last_became_active":"2024-04-26T21:10:03.713380+0000","last_became_peered":"2024-04-26T21:10:03.713380+0000","last_unstale":"2024-04-26T21:10:03.714910+0000","last_undegraded":"2024-04-26T21:10:03.714910+0000","last_fullsized":"2024-04-26T21:10:03.714910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:43.726445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707897+0000","last_change":"2024-04-26T21:09:41.070626+0000","last_active":"2024-04-26T21:10:03.707897+0000","last_peered":"2024-04-26T21:10:03.707897+0000","last_clean":"2024-04-26T21:10:03.707897+0000","last_became_active":"2024-04-26T21:09:41.070258+0000","last_became_peered":"2024-04-26T21:09:41.070258+0000","last_unstale":"2024-04-26T21:10:03.707897+0000","last_undegraded":"2024-04-26T21:10:03.707897+0000","last_fullsized":"2024-04-26T21:10:03.707897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:10.491491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.714363+0000","last_change":"2024-04-26T21:10:03.714363+0000","last_active":"2024-04-26T21:10:03.714363+0000","last_peered":"2024-04-26T21:10:03.714363+0000","last_clean":"2024-04-26T21:10:03.714363+0000","last_became_active":"2024-04-26T21:10:03.713233+0000","last_became_peered":"2024-04-26T21:10:03.713233+0000","last_unstale":"2024-04-26T21:10:03.714363+0000","last_undegraded":"2024-04-26T21:10:03.714363+0000","last_fullsized":"2024-04-26T21:10:03.714363+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.010890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690314+0000","last_change":"2024-04-26T21:09:41.069684+0000","last_active":"2024-04-26T21:10:03.690314+0000","last_peered":"2024-04-26T21:10:03.690314+0000","last_clean":"2024-04-26T21:10:03.690314+0000","last_became_active":"2024-04-26T21:09:41.069323+0000","last_became_peered":"2024-04-26T21:09:41.069323+0000","last_unstale":"2024-04-26T21:10:03.690314+0000","last_undegraded":"2024-04-26T21:10:03.690314+0000","last_fullsized":"2024-04-26T21:10:03.690314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:57.122606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.695833+0000","last_change":"2024-04-26T21:09:41.069130+0000","last_active":"2024-04-26T21:10:03.695833+0000","last_peered":"2024-04-26T21:10:03.695833+0000","last_clean":"2024-04-26T21:10:03.695833+0000","last_became_active":"2024-04-26T21:09:41.068682+0000","last_became_peered":"2024-04-26T21:09:41.068682+0000","last_unstale":"2024-04-26T21:10:03.695833+0000","last_undegraded":"2024-04-26T21:10:03.695833+0000","last_fullsized":"2024-04-26T21:10:03.695833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:39.389562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699230+0000","last_change":"2024-04-26T21:09:41.073930+0000","last_active":"2024-04-26T21:10:03.699230+0000","last_peered":"2024-04-26T21:10:03.699230+0000","last_clean":"2024-04-26T21:10:03.699230+0000","last_became_active":"2024-04-26T21:09:41.072566+0000","last_became_peered":"2024-04-26T21:09:41.072566+0000","last_unstale":"2024-04-26T21:10:03.699230+0000","last_undegraded":"2024-04-26T21:10:03.699230+0000","last_fullsized":"2024-04-26T21:10:03.699230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:48.613310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700662+0000","last_change":"2024-04-26T21:09:41.065598+0000","last_active":"2024-04-26T21:10:03.700662+0000","last_peered":"2024-04-26T21:10:03.700662+0000","last_clean":"2024-04-26T21:10:03.700662+0000","last_became_active":"2024-04-26T21:09:41.064977+0000","last_became_peered":"2024-04-26T21:09:41.064977+0000","last_unstale":"2024-04-26T21:10:03.700662+0000","last_undegraded":"2024-04-26T21:10:03.700662+0000","last_fullsized":"2024-04-26T21:10:03.700662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:39.776967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690907+0000","last_change":"2024-04-26T21:09:41.077939+0000","last_active":"2024-04-26T21:10:03.690907+0000","last_peered":"2024-04-26T21:10:03.690907+0000","last_clean":"2024-04-26T21:10:03.690907+0000","last_became_active":"2024-04-26T21:09:41.077641+0000","last_became_peered":"2024-04-26T21:09:41.077641+0000","last_unstale":"2024-04-26T21:10:03.690907+0000","last_undegraded":"2024-04-26T21:10:03.690907+0000","last_fullsized":"2024-04-26T21:10:03.690907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:47.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719794+0000","last_change":"2024-04-26T21:09:41.075109+0000","last_active":"2024-04-26T21:10:03.719794+0000","last_peered":"2024-04-26T21:10:03.719794+0000","last_clean":"2024-04-26T21:10:03.719794+0000","last_became_active":"2024-04-26T21:09:41.073866+0000","last_became_peered":"2024-04-26T21:09:41.073866+0000","last_unstale":"2024-04-26T21:10:03.719794+0000","last_undegraded":"2024-04-26T21:10:03.719794+0000","last_fullsized":"2024-04-26T21:10:03.719794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:16.786155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692482+0000","last_change":"2024-04-26T21:09:41.074934+0000","last_active":"2024-04-26T21:10:03.692482+0000","last_peered":"2024-04-26T21:10:03.692482+0000","last_clean":"2024-04-26T21:10:03.692482+0000","last_became_active":"2024-04-26T21:09:41.072388+0000","last_became_peered":"2024-04-26T21:09:41.072388+0000","last_unstale":"2024-04-26T21:10:03.692482+0000","last_undegraded":"2024-04-26T21:10:03.692482+0000","last_fullsized":"2024-04-26T21:10:03.692482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:23.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716459+0000","last_change":"2024-04-26T21:09:41.062598+0000","last_active":"2024-04-26T21:10:03.716459+0000","last_peered":"2024-04-26T21:10:03.716459+0000","last_clean":"2024-04-26T21:10:03.716459+0000","last_became_active":"2024-04-26T21:09:41.061828+0000","last_became_peered":"2024-04-26T21:09:41.061828+0000","last_unstale":"2024-04-26T21:10:03.716459+0000","last_undegraded":"2024-04-26T21:10:03.716459+0000","last_fullsized":"2024-04-26T21:10:03.716459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:52.877573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716269+0000","last_change":"2024-04-26T21:09:41.064874+0000","last_active":"2024-04-26T21:10:03.716269+0000","last_peered":"2024-04-26T21:10:03.716269+0000","last_clean":"2024-04-26T21:10:03.716269+0000","last_became_active":"2024-04-26T21:09:41.060190+0000","last_became_peered":"2024-04-26T21:09:41.060190+0000","last_unstale":"2024-04-26T21:10:03.716269+0000","last_undegraded":"2024-04-26T21:10:03.716269+0000","last_fullsized":"2024-04-26T21:10:03.716269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697003+0000","last_change":"2024-04-26T21:09:41.067343+0000","last_active":"2024-04-26T21:10:03.697003+0000","last_peered":"2024-04-26T21:10:03.697003+0000","last_clean":"2024-04-26T21:10:03.697003+0000","last_became_active":"2024-04-26T21:09:41.066248+0000","last_became_peered":"2024-04-26T21:09:41.066248+0000","last_unstale":"2024-04-26T21:10:03.697003+0000","last_undegraded":"2024-04-26T21:10:03.697003+0000","last_fullsized":"2024-04-26T21:10:03.697003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:48:40.476089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697120+0000","last_change":"2024-04-26T21:09:41.079414+0000","last_active":"2024-04-26T21:10:03.697120+0000","last_peered":"2024-04-26T21:10:03.697120+0000","last_clean":"2024-04-26T21:10:03.697120+0000","last_became_active":"2024-04-26T21:09:41.078949+0000","last_became_peered":"2024-04-26T21:09:41.078949+0000","last_unstale":"2024-04-26T21:10:03.697120+0000","last_undegraded":"2024-04-26T21:10:03.697120+0000","last_fullsized":"2024-04-26T21:10:03.697120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:19.947290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697532+0000","last_change":"2024-04-26T21:09:41.069376+0000","last_active":"2024-04-26T21:10:03.697532+0000","last_peered":"2024-04-26T21:10:03.697532+0000","last_clean":"2024-04-26T21:10:03.697532+0000","last_became_active":"2024-04-26T21:09:41.069049+0000","last_became_peered":"2024-04-26T21:09:41.069049+0000","last_unstale":"2024-04-26T21:10:03.697532+0000","last_undegraded":"2024-04-26T21:10:03.697532+0000","last_fullsized":"2024-04-26T21:10:03.697532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:25.058292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.704488+0000","last_change":"2024-04-26T21:09:41.072485+0000","last_active":"2024-04-26T21:10:03.704488+0000","last_peered":"2024-04-26T21:10:03.704488+0000","last_clean":"2024-04-26T21:10:03.704488+0000","last_became_active":"2024-04-26T21:09:41.070990+0000","last_became_peered":"2024-04-26T21:09:41.070990+0000","last_unstale":"2024-04-26T21:10:03.704488+0000","last_undegraded":"2024-04-26T21:10:03.704488+0000","last_fullsized":"2024-04-26T21:10:03.704488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:40.648097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":144,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:12.896564+0000","last_change":"2024-04-26T21:10:03.739254+0000","last_active":"2024-04-26T21:11:12.896564+0000","last_peered":"2024-04-26T21:11:12.896564+0000","last_clean":"2024-04-26T21:11:12.896564+0000","last_became_active":"2024-04-26T21:10:03.709954+0000","last_became_peered":"2024-04-26T21:10:03.709954+0000","last_unstale":"2024-04-26T21:11:12.896564+0000","last_undegraded":"2024-04-26T21:11:12.896564+0000","last_fullsized":"2024-04-26T21:11:12.896564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:33.358811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698762+0000","last_change":"2024-04-26T21:09:41.084249+0000","last_active":"2024-04-26T21:10:03.698762+0000","last_peered":"2024-04-26T21:10:03.698762+0000","last_clean":"2024-04-26T21:10:03.698762+0000","last_became_active":"2024-04-26T21:09:41.083580+0000","last_became_peered":"2024-04-26T21:09:41.083580+0000","last_unstale":"2024-04-26T21:10:03.698762+0000","last_undegraded":"2024-04-26T21:10:03.698762+0000","last_fullsized":"2024-04-26T21:10:03.698762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:02.615403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699085+0000","last_change":"2024-04-26T21:09:40.389376+0000","last_active":"2024-04-26T21:10:03.699085+0000","last_peered":"2024-04-26T21:10:03.699085+0000","last_clean":"2024-04-26T21:10:03.699085+0000","last_became_active":"2024-04-26T21:09:40.388761+0000","last_became_peered":"2024-04-26T21:09:40.388761+0000","last_unstale":"2024-04-26T21:10:03.699085+0000","last_undegraded":"2024-04-26T21:10:03.699085+0000","last_fullsized":"2024-04-26T21:10:03.699085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:39.940524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716712+0000","last_change":"2024-04-26T21:09:41.073193+0000","last_active":"2024-04-26T21:10:03.716712+0000","last_peered":"2024-04-26T21:10:03.716712+0000","last_clean":"2024-04-26T21:10:03.716712+0000","last_became_active":"2024-04-26T21:09:41.071569+0000","last_became_peered":"2024-04-26T21:09:41.071569+0000","last_unstale":"2024-04-26T21:10:03.716712+0000","last_undegraded":"2024-04-26T21:10:03.716712+0000","last_fullsized":"2024-04-26T21:10:03.716712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:40.078386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.708560+0000","last_change":"2024-04-26T21:09:40.394188+0000","last_active":"2024-04-26T21:10:03.708560+0000","last_peered":"2024-04-26T21:10:03.708560+0000","last_clean":"2024-04-26T21:10:03.708560+0000","last_became_active":"2024-04-26T21:09:40.393854+0000","last_became_peered":"2024-04-26T21:09:40.393854+0000","last_unstale":"2024-04-26T21:10:03.708560+0000","last_undegraded":"2024-04-26T21:10:03.708560+0000","last_fullsized":"2024-04-26T21:10:03.708560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:30.232353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697833+0000","last_change":"2024-04-26T21:09:41.068545+0000","last_active":"2024-04-26T21:10:03.697833+0000","last_peered":"2024-04-26T21:10:03.697833+0000","last_clean":"2024-04-26T21:10:03.697833+0000","last_became_active":"2024-04-26T21:09:41.066805+0000","last_became_peered":"2024-04-26T21:09:41.066805+0000","last_unstale":"2024-04-26T21:10:03.697833+0000","last_undegraded":"2024-04-26T21:10:03.697833+0000","last_fullsized":"2024-04-26T21:10:03.697833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:32.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.709517+0000","last_change":"2024-04-26T21:09:40.382727+0000","last_active":"2024-04-26T21:10:03.709517+0000","last_peered":"2024-04-26T21:10:03.709517+0000","last_clean":"2024-04-26T21:10:03.709517+0000","last_became_active":"2024-04-26T21:09:40.380205+0000","last_became_peered":"2024-04-26T21:09:40.380205+0000","last_unstale":"2024-04-26T21:10:03.709517+0000","last_undegraded":"2024-04-26T21:10:03.709517+0000","last_fullsized":"2024-04-26T21:10:03.709517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:32.981478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.722050+0000","last_change":"2024-04-26T21:10:03.721385+0000","last_active":"2024-04-26T21:10:03.722050+0000","last_peered":"2024-04-26T21:10:03.722050+0000","last_clean":"2024-04-26T21:10:03.722050+0000","last_became_active":"2024-04-26T21:10:03.721009+0000","last_became_peered":"2024-04-26T21:10:03.721009+0000","last_unstale":"2024-04-26T21:10:03.722050+0000","last_undegraded":"2024-04-26T21:10:03.722050+0000","last_fullsized":"2024-04-26T21:10:03.722050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:25.240987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:13.739548+0000","last_change":"2024-04-26T21:09:40.395282+0000","last_active":"2024-04-26T21:11:13.739548+0000","last_peered":"2024-04-26T21:11:13.739548+0000","last_clean":"2024-04-26T21:11:13.739548+0000","last_became_active":"2024-04-26T21:09:40.394858+0000","last_became_peered":"2024-04-26T21:09:40.394858+0000","last_unstale":"2024-04-26T21:11:13.739548+0000","last_undegraded":"2024-04-26T21:11:13.739548+0000","last_fullsized":"2024-04-26T21:11:13.739548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:38.330771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.709417+0000","last_change":"2024-04-26T21:09:41.064853+0000","last_active":"2024-04-26T21:10:03.709417+0000","last_peered":"2024-04-26T21:10:03.709417+0000","last_clean":"2024-04-26T21:10:03.709417+0000","last_became_active":"2024-04-26T21:09:41.064236+0000","last_became_peered":"2024-04-26T21:09:41.064236+0000","last_unstale":"2024-04-26T21:10:03.709417+0000","last_undegraded":"2024-04-26T21:10:03.709417+0000","last_fullsized":"2024-04-26T21:10:03.709417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:05.549027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700407+0000","last_change":"2024-04-26T21:09:40.390857+0000","last_active":"2024-04-26T21:10:03.700407+0000","last_peered":"2024-04-26T21:10:03.700407+0000","last_clean":"2024-04-26T21:10:03.700407+0000","last_became_active":"2024-04-26T21:09:40.390513+0000","last_became_peered":"2024-04-26T21:09:40.390513+0000","last_unstale":"2024-04-26T21:10:03.700407+0000","last_undegraded":"2024-04-26T21:10:03.700407+0000","last_fullsized":"2024-04-26T21:10:03.700407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:52.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700524+0000","last_change":"2024-04-26T21:10:03.700524+0000","last_active":"2024-04-26T21:10:03.700524+0000","last_peered":"2024-04-26T21:10:03.700524+0000","last_clean":"2024-04-26T21:10:03.700524+0000","last_became_active":"2024-04-26T21:10:03.699371+0000","last_became_peered":"2024-04-26T21:10:03.699371+0000","last_unstale":"2024-04-26T21:10:03.700524+0000","last_undegraded":"2024-04-26T21:10:03.700524+0000","last_fullsized":"2024-04-26T21:10:03.700524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:22.223665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691275+0000","last_change":"2024-04-26T21:09:40.390893+0000","last_active":"2024-04-26T21:10:03.691275+0000","last_peered":"2024-04-26T21:10:03.691275+0000","last_clean":"2024-04-26T21:10:03.691275+0000","last_became_active":"2024-04-26T21:09:40.390519+0000","last_became_peered":"2024-04-26T21:09:40.390519+0000","last_unstale":"2024-04-26T21:10:03.691275+0000","last_undegraded":"2024-04-26T21:10:03.691275+0000","last_fullsized":"2024-04-26T21:10:03.691275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:40.963614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698119+0000","last_change":"2024-04-26T21:09:41.074751+0000","last_active":"2024-04-26T21:10:03.698119+0000","last_peered":"2024-04-26T21:10:03.698119+0000","last_clean":"2024-04-26T21:10:03.698119+0000","last_became_active":"2024-04-26T21:09:41.074330+0000","last_became_peered":"2024-04-26T21:09:41.074330+0000","last_unstale":"2024-04-26T21:10:03.698119+0000","last_undegraded":"2024-04-26T21:10:03.698119+0000","last_fullsized":"2024-04-26T21:10:03.698119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:47.759543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690977+0000","last_change":"2024-04-26T21:09:40.389971+0000","last_active":"2024-04-26T21:10:03.690977+0000","last_peered":"2024-04-26T21:10:03.690977+0000","last_clean":"2024-04-26T21:10:03.690977+0000","last_became_active":"2024-04-26T21:09:40.389093+0000","last_became_peered":"2024-04-26T21:09:40.389093+0000","last_unstale":"2024-04-26T21:10:03.690977+0000","last_undegraded":"2024-04-26T21:10:03.690977+0000","last_fullsized":"2024-04-26T21:10:03.690977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:18.838994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698551+0000","last_change":"2024-04-26T21:09:41.068903+0000","last_active":"2024-04-26T21:10:03.698551+0000","last_peered":"2024-04-26T21:10:03.698551+0000","last_clean":"2024-04-26T21:10:03.698551+0000","last_became_active":"2024-04-26T21:09:41.068585+0000","last_became_peered":"2024-04-26T21:09:41.068585+0000","last_unstale":"2024-04-26T21:10:03.698551+0000","last_undegraded":"2024-04-26T21:10:03.698551+0000","last_fullsized":"2024-04-26T21:10:03.698551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:18.372213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":189,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.729225+0000","last_change":"2024-04-26T21:09:40.391501+0000","last_active":"2024-04-26T21:11:10.729225+0000","last_peered":"2024-04-26T21:11:10.729225+0000","last_clean":"2024-04-26T21:11:10.729225+0000","last_became_active":"2024-04-26T21:09:40.390434+0000","last_became_peered":"2024-04-26T21:09:40.390434+0000","last_unstale":"2024-04-26T21:11:10.729225+0000","last_undegraded":"2024-04-26T21:11:10.729225+0000","last_fullsized":"2024-04-26T21:11:10.729225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:11.854716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698702+0000","last_change":"2024-04-26T21:09:41.064937+0000","last_active":"2024-04-26T21:10:03.698702+0000","last_peered":"2024-04-26T21:10:03.698702+0000","last_clean":"2024-04-26T21:10:03.698702+0000","last_became_active":"2024-04-26T21:09:41.063667+0000","last_became_peered":"2024-04-26T21:09:41.063667+0000","last_unstale":"2024-04-26T21:10:03.698702+0000","last_undegraded":"2024-04-26T21:10:03.698702+0000","last_fullsized":"2024-04-26T21:10:03.698702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:24.380869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.765337+0000","last_change":"2024-04-26T21:10:03.765337+0000","last_active":"2024-04-26T21:10:03.765337+0000","last_peered":"2024-04-26T21:10:03.765337+0000","last_clean":"2024-04-26T21:10:03.765337+0000","last_became_active":"2024-04-26T21:10:03.722557+0000","last_became_peered":"2024-04-26T21:10:03.722557+0000","last_unstale":"2024-04-26T21:10:03.765337+0000","last_undegraded":"2024-04-26T21:10:03.765337+0000","last_fullsized":"2024-04-26T21:10:03.765337+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:34.370015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716620+0000","last_change":"2024-04-26T21:09:41.064559+0000","last_active":"2024-04-26T21:10:03.716620+0000","last_peered":"2024-04-26T21:10:03.716620+0000","last_clean":"2024-04-26T21:10:03.716620+0000","last_became_active":"2024-04-26T21:09:41.063117+0000","last_became_peered":"2024-04-26T21:09:41.063117+0000","last_unstale":"2024-04-26T21:10:03.716620+0000","last_undegraded":"2024-04-26T21:10:03.716620+0000","last_fullsized":"2024-04-26T21:10:03.716620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:34.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692110+0000","last_change":"2024-04-26T21:09:40.390507+0000","last_active":"2024-04-26T21:10:03.692110+0000","last_peered":"2024-04-26T21:10:03.692110+0000","last_clean":"2024-04-26T21:10:03.692110+0000","last_became_active":"2024-04-26T21:09:40.388503+0000","last_became_peered":"2024-04-26T21:09:40.388503+0000","last_unstale":"2024-04-26T21:10:03.692110+0000","last_undegraded":"2024-04-26T21:10:03.692110+0000","last_fullsized":"2024-04-26T21:10:03.692110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:40.432667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697601+0000","last_change":"2024-04-26T21:09:41.065188+0000","last_active":"2024-04-26T21:10:03.697601+0000","last_peered":"2024-04-26T21:10:03.697601+0000","last_clean":"2024-04-26T21:10:03.697601+0000","last_became_active":"2024-04-26T21:09:41.061218+0000","last_became_peered":"2024-04-26T21:09:41.061218+0000","last_unstale":"2024-04-26T21:10:03.697601+0000","last_undegraded":"2024-04-26T21:10:03.697601+0000","last_fullsized":"2024-04-26T21:10:03.697601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:42.087561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":242,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.927758+0000","last_change":"2024-04-26T21:10:03.756099+0000","last_active":"2024-04-26T21:11:10.927758+0000","last_peered":"2024-04-26T21:11:10.927758+0000","last_clean":"2024-04-26T21:11:10.927758+0000","last_became_active":"2024-04-26T21:10:03.723285+0000","last_became_peered":"2024-04-26T21:10:03.723285+0000","last_unstale":"2024-04-26T21:11:10.927758+0000","last_undegraded":"2024-04-26T21:11:10.927758+0000","last_fullsized":"2024-04-26T21:11:10.927758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:19.825308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":128,"num_write":31,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":48166,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707938+0000","last_change":"2024-04-26T21:09:41.063711+0000","last_active":"2024-04-26T21:10:03.707938+0000","last_peered":"2024-04-26T21:10:03.707938+0000","last_clean":"2024-04-26T21:10:03.707938+0000","last_became_active":"2024-04-26T21:09:41.063330+0000","last_became_peered":"2024-04-26T21:09:41.063330+0000","last_unstale":"2024-04-26T21:10:03.707938+0000","last_undegraded":"2024-04-26T21:10:03.707938+0000","last_fullsized":"2024-04-26T21:10:03.707938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:28.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697890+0000","last_change":"2024-04-26T21:09:40.404407+0000","last_active":"2024-04-26T21:10:03.697890+0000","last_peered":"2024-04-26T21:10:03.697890+0000","last_clean":"2024-04-26T21:10:03.697890+0000","last_became_active":"2024-04-26T21:09:40.402751+0000","last_became_peered":"2024-04-26T21:09:40.402751+0000","last_unstale":"2024-04-26T21:10:03.697890+0000","last_undegraded":"2024-04-26T21:10:03.697890+0000","last_fullsized":"2024-04-26T21:10:03.697890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:32.171086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691862+0000","last_change":"2024-04-26T21:09:41.079310+0000","last_active":"2024-04-26T21:10:03.691862+0000","last_peered":"2024-04-26T21:10:03.691862+0000","last_clean":"2024-04-26T21:10:03.691862+0000","last_became_active":"2024-04-26T21:09:41.077848+0000","last_became_peered":"2024-04-26T21:09:41.077848+0000","last_unstale":"2024-04-26T21:10:03.691862+0000","last_undegraded":"2024-04-26T21:10:03.691862+0000","last_fullsized":"2024-04-26T21:10:03.691862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:51.745349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691062+0000","last_change":"2024-04-26T21:09:40.391216+0000","last_active":"2024-04-26T21:10:03.691062+0000","last_peered":"2024-04-26T21:10:03.691062+0000","last_clean":"2024-04-26T21:10:03.691062+0000","last_became_active":"2024-04-26T21:09:40.390753+0000","last_became_peered":"2024-04-26T21:09:40.390753+0000","last_unstale":"2024-04-26T21:10:03.691062+0000","last_undegraded":"2024-04-26T21:10:03.691062+0000","last_fullsized":"2024-04-26T21:10:03.691062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:24.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690737+0000","last_change":"2024-04-26T21:09:41.079881+0000","last_active":"2024-04-26T21:10:03.690737+0000","last_peered":"2024-04-26T21:10:03.690737+0000","last_clean":"2024-04-26T21:10:03.690737+0000","last_became_active":"2024-04-26T21:09:41.079506+0000","last_became_peered":"2024-04-26T21:09:41.079506+0000","last_unstale":"2024-04-26T21:10:03.690737+0000","last_undegraded":"2024-04-26T21:10:03.690737+0000","last_fullsized":"2024-04-26T21:10:03.690737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:56.300294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":111,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.729231+0000","last_change":"2024-04-26T21:09:40.391458+0000","last_active":"2024-04-26T21:11:10.729231+0000","last_peered":"2024-04-26T21:11:10.729231+0000","last_clean":"2024-04-26T21:11:10.729231+0000","last_became_active":"2024-04-26T21:09:40.388494+0000","last_became_peered":"2024-04-26T21:09:40.388494+0000","last_unstale":"2024-04-26T21:11:10.729231+0000","last_undegraded":"2024-04-26T21:11:10.729231+0000","last_fullsized":"2024-04-26T21:11:10.729231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:07.523885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690197+0000","last_change":"2024-04-26T21:09:41.068278+0000","last_active":"2024-04-26T21:10:03.690197+0000","last_peered":"2024-04-26T21:10:03.690197+0000","last_clean":"2024-04-26T21:10:03.690197+0000","last_became_active":"2024-04-26T21:09:41.067865+0000","last_became_peered":"2024-04-26T21:09:41.067865+0000","last_unstale":"2024-04-26T21:10:03.690197+0000","last_undegraded":"2024-04-26T21:10:03.690197+0000","last_fullsized":"2024-04-26T21:10:03.690197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:19.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707555+0000","last_change":"2024-04-26T21:09:40.394390+0000","last_active":"2024-04-26T21:10:03.707555+0000","last_peered":"2024-04-26T21:10:03.707555+0000","last_clean":"2024-04-26T21:10:03.707555+0000","last_became_active":"2024-04-26T21:09:40.389728+0000","last_became_peered":"2024-04-26T21:09:40.389728+0000","last_unstale":"2024-04-26T21:10:03.707555+0000","last_undegraded":"2024-04-26T21:10:03.707555+0000","last_fullsized":"2024-04-26T21:10:03.707555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:42.301974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690944+0000","last_change":"2024-04-26T21:09:41.075703+0000","last_active":"2024-04-26T21:10:03.690944+0000","last_peered":"2024-04-26T21:10:03.690944+0000","last_clean":"2024-04-26T21:10:03.690944+0000","last_became_active":"2024-04-26T21:09:41.075310+0000","last_became_peered":"2024-04-26T21:09:41.075310+0000","last_unstale":"2024-04-26T21:10:03.690944+0000","last_undegraded":"2024-04-26T21:10:03.690944+0000","last_fullsized":"2024-04-26T21:10:03.690944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:20.030941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.710094+0000","last_change":"2024-04-26T21:09:40.381293+0000","last_active":"2024-04-26T21:10:03.710094+0000","last_peered":"2024-04-26T21:10:03.710094+0000","last_clean":"2024-04-26T21:10:03.710094+0000","last_became_active":"2024-04-26T21:09:40.380287+0000","last_became_peered":"2024-04-26T21:09:40.380287+0000","last_unstale":"2024-04-26T21:10:03.710094+0000","last_undegraded":"2024-04-26T21:10:03.710094+0000","last_fullsized":"2024-04-26T21:10:03.710094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:47.479546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698460+0000","last_change":"2024-04-26T21:09:41.077928+0000","last_active":"2024-04-26T21:10:03.698460+0000","last_peered":"2024-04-26T21:10:03.698460+0000","last_clean":"2024-04-26T21:10:03.698460+0000","last_became_active":"2024-04-26T21:09:41.076611+0000","last_became_peered":"2024-04-26T21:09:41.076611+0000","last_unstale":"2024-04-26T21:10:03.698460+0000","last_undegraded":"2024-04-26T21:10:03.698460+0000","last_fullsized":"2024-04-26T21:10:03.698460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697596+0000","last_change":"2024-04-26T21:09:40.394161+0000","last_active":"2024-04-26T21:10:03.697596+0000","last_peered":"2024-04-26T21:10:03.697596+0000","last_clean":"2024-04-26T21:10:03.697596+0000","last_became_active":"2024-04-26T21:09:40.393277+0000","last_became_peered":"2024-04-26T21:09:40.393277+0000","last_unstale":"2024-04-26T21:10:03.697596+0000","last_undegraded":"2024-04-26T21:10:03.697596+0000","last_fullsized":"2024-04-26T21:10:03.697596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:32.408080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701600+0000","last_change":"2024-04-26T21:09:41.070926+0000","last_active":"2024-04-26T21:10:03.701600+0000","last_peered":"2024-04-26T21:10:03.701600+0000","last_clean":"2024-04-26T21:10:03.701600+0000","last_became_active":"2024-04-26T21:09:41.070260+0000","last_became_peered":"2024-04-26T21:09:41.070260+0000","last_unstale":"2024-04-26T21:10:03.701600+0000","last_undegraded":"2024-04-26T21:10:03.701600+0000","last_fullsized":"2024-04-26T21:10:03.701600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:22.916960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:09.889969+0000","last_change":"2024-04-26T21:09:40.395088+0000","last_active":"2024-04-26T21:11:09.889969+0000","last_peered":"2024-04-26T21:11:09.889969+0000","last_clean":"2024-04-26T21:11:09.889969+0000","last_became_active":"2024-04-26T21:09:40.393921+0000","last_became_peered":"2024-04-26T21:09:40.393921+0000","last_unstale":"2024-04-26T21:11:09.889969+0000","last_undegraded":"2024-04-26T21:11:09.889969+0000","last_fullsized":"2024-04-26T21:11:09.889969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:30.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701596+0000","last_change":"2024-04-26T21:09:41.064834+0000","last_active":"2024-04-26T21:10:03.701596+0000","last_peered":"2024-04-26T21:10:03.701596+0000","last_clean":"2024-04-26T21:10:03.701596+0000","last_became_active":"2024-04-26T21:09:41.063875+0000","last_became_peered":"2024-04-26T21:09:41.063875+0000","last_unstale":"2024-04-26T21:10:03.701596+0000","last_undegraded":"2024-04-26T21:10:03.701596+0000","last_fullsized":"2024-04-26T21:10:03.701596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:48.438740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691737+0000","last_change":"2024-04-26T21:09:41.075181+0000","last_active":"2024-04-26T21:10:03.691737+0000","last_peered":"2024-04-26T21:10:03.691737+0000","last_clean":"2024-04-26T21:10:03.691737+0000","last_became_active":"2024-04-26T21:09:41.073137+0000","last_became_peered":"2024-04-26T21:09:41.073137+0000","last_unstale":"2024-04-26T21:10:03.691737+0000","last_undegraded":"2024-04-26T21:10:03.691737+0000","last_fullsized":"2024-04-26T21:10:03.691737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:08.609048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:11.730577+0000","last_change":"2024-04-26T21:09:40.404159+0000","last_active":"2024-04-26T21:11:11.730577+0000","last_peered":"2024-04-26T21:11:11.730577+0000","last_clean":"2024-04-26T21:11:11.730577+0000","last_became_active":"2024-04-26T21:09:40.403828+0000","last_became_peered":"2024-04-26T21:09:40.403828+0000","last_unstale":"2024-04-26T21:11:11.730577+0000","last_undegraded":"2024-04-26T21:11:11.730577+0000","last_fullsized":"2024-04-26T21:11:11.730577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:51.245405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698753+0000","last_change":"2024-04-26T21:09:41.068913+0000","last_active":"2024-04-26T21:10:03.698753+0000","last_peered":"2024-04-26T21:10:03.698753+0000","last_clean":"2024-04-26T21:10:03.698753+0000","last_became_active":"2024-04-26T21:09:41.068534+0000","last_became_peered":"2024-04-26T21:09:41.068534+0000","last_unstale":"2024-04-26T21:10:03.698753+0000","last_undegraded":"2024-04-26T21:10:03.698753+0000","last_fullsized":"2024-04-26T21:10:03.698753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:39.739853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719296+0000","last_change":"2024-04-26T21:09:40.387055+0000","last_active":"2024-04-26T21:10:03.719296+0000","last_peered":"2024-04-26T21:10:03.719296+0000","last_clean":"2024-04-26T21:10:03.719296+0000","last_became_active":"2024-04-26T21:09:40.386684+0000","last_became_peered":"2024-04-26T21:09:40.386684+0000","last_unstale":"2024-04-26T21:10:03.719296+0000","last_undegraded":"2024-04-26T21:10:03.719296+0000","last_fullsized":"2024-04-26T21:10:03.719296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:02.871494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.690520+0000","last_change":"2024-04-26T21:08:05.429394+0000","last_active":"2024-04-26T21:10:03.690520+0000","last_peered":"2024-04-26T21:10:03.690520+0000","last_clean":"2024-04-26T21:10:03.690520+0000","last_became_active":"2024-04-26T21:08:05.428055+0000","last_became_peered":"2024-04-26T21:08:05.428055+0000","last_unstale":"2024-04-26T21:10:03.690520+0000","last_undegraded":"2024-04-26T21:10:03.690520+0000","last_fullsized":"2024-04-26T21:10:03.690520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:03:22.595431+0000","last_clean_scrub_stamp":"2024-04-26T21:03:22.595431+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:23.104153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691138+0000","last_change":"2024-04-26T21:09:41.080213+0000","last_active":"2024-04-26T21:10:03.691138+0000","last_peered":"2024-04-26T21:10:03.691138+0000","last_clean":"2024-04-26T21:10:03.691138+0000","last_became_active":"2024-04-26T21:09:41.076313+0000","last_became_peered":"2024-04-26T21:09:41.076313+0000","last_unstale":"2024-04-26T21:10:03.691138+0000","last_undegraded":"2024-04-26T21:10:03.691138+0000","last_fullsized":"2024-04-26T21:10:03.691138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:48.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691598+0000","last_change":"2024-04-26T21:09:40.387563+0000","last_active":"2024-04-26T21:10:03.691598+0000","last_peered":"2024-04-26T21:10:03.691598+0000","last_clean":"2024-04-26T21:10:03.691598+0000","last_became_active":"2024-04-26T21:09:40.383243+0000","last_became_peered":"2024-04-26T21:09:40.383243+0000","last_unstale":"2024-04-26T21:10:03.691598+0000","last_undegraded":"2024-04-26T21:10:03.691598+0000","last_fullsized":"2024-04-26T21:10:03.691598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:16.905074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.708245+0000","last_change":"2024-04-26T21:09:40.398078+0000","last_active":"2024-04-26T21:10:03.708245+0000","last_peered":"2024-04-26T21:10:03.708245+0000","last_clean":"2024-04-26T21:10:03.708245+0000","last_became_active":"2024-04-26T21:09:40.397007+0000","last_became_peered":"2024-04-26T21:09:40.397007+0000","last_unstale":"2024-04-26T21:10:03.708245+0000","last_undegraded":"2024-04-26T21:10:03.708245+0000","last_fullsized":"2024-04-26T21:10:03.708245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:18.374678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692204+0000","last_change":"2024-04-26T21:09:41.063430+0000","last_active":"2024-04-26T21:10:03.692204+0000","last_peered":"2024-04-26T21:10:03.692204+0000","last_clean":"2024-04-26T21:10:03.692204+0000","last_became_active":"2024-04-26T21:09:41.062591+0000","last_became_peered":"2024-04-26T21:09:41.062591+0000","last_unstale":"2024-04-26T21:10:03.692204+0000","last_undegraded":"2024-04-26T21:10:03.692204+0000","last_fullsized":"2024-04-26T21:10:03.692204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:58.874576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697875+0000","last_change":"2024-04-26T21:09:40.378465+0000","last_active":"2024-04-26T21:10:03.697875+0000","last_peered":"2024-04-26T21:10:03.697875+0000","last_clean":"2024-04-26T21:10:03.697875+0000","last_became_active":"2024-04-26T21:09:40.378089+0000","last_became_peered":"2024-04-26T21:09:40.378089+0000","last_unstale":"2024-04-26T21:10:03.697875+0000","last_undegraded":"2024-04-26T21:10:03.697875+0000","last_fullsized":"2024-04-26T21:10:03.697875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:56.425077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699449+0000","last_change":"2024-04-26T21:09:41.080446+0000","last_active":"2024-04-26T21:10:03.699449+0000","last_peered":"2024-04-26T21:10:03.699449+0000","last_clean":"2024-04-26T21:10:03.699449+0000","last_became_active":"2024-04-26T21:09:41.074363+0000","last_became_peered":"2024-04-26T21:09:41.074363+0000","last_unstale":"2024-04-26T21:10:03.699449+0000","last_undegraded":"2024-04-26T21:10:03.699449+0000","last_fullsized":"2024-04-26T21:10:03.699449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:14.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.719194+0000","last_change":"2024-04-26T21:09:40.403106+0000","last_active":"2024-04-26T21:10:03.719194+0000","last_peered":"2024-04-26T21:10:03.719194+0000","last_clean":"2024-04-26T21:10:03.719194+0000","last_became_active":"2024-04-26T21:09:40.402630+0000","last_became_peered":"2024-04-26T21:09:40.402630+0000","last_unstale":"2024-04-26T21:10:03.719194+0000","last_undegraded":"2024-04-26T21:10:03.719194+0000","last_fullsized":"2024-04-26T21:10:03.719194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:37.555164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716417+0000","last_change":"2024-04-26T21:09:41.069407+0000","last_active":"2024-04-26T21:10:03.716417+0000","last_peered":"2024-04-26T21:10:03.716417+0000","last_clean":"2024-04-26T21:10:03.716417+0000","last_became_active":"2024-04-26T21:09:41.067522+0000","last_became_peered":"2024-04-26T21:09:41.067522+0000","last_unstale":"2024-04-26T21:10:03.716417+0000","last_undegraded":"2024-04-26T21:10:03.716417+0000","last_fullsized":"2024-04-26T21:10:03.716417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.338665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.696784+0000","last_change":"2024-04-26T21:09:40.391323+0000","last_active":"2024-04-26T21:10:03.696784+0000","last_peered":"2024-04-26T21:10:03.696784+0000","last_clean":"2024-04-26T21:10:03.696784+0000","last_became_active":"2024-04-26T21:09:40.390982+0000","last_became_peered":"2024-04-26T21:09:40.390982+0000","last_unstale":"2024-04-26T21:10:03.696784+0000","last_undegraded":"2024-04-26T21:10:03.696784+0000","last_fullsized":"2024-04-26T21:10:03.696784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:03.465013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.707438+0000","last_change":"2024-04-26T21:09:41.068305+0000","last_active":"2024-04-26T21:10:03.707438+0000","last_peered":"2024-04-26T21:10:03.707438+0000","last_clean":"2024-04-26T21:10:03.707438+0000","last_became_active":"2024-04-26T21:09:41.067946+0000","last_became_peered":"2024-04-26T21:09:41.067946+0000","last_unstale":"2024-04-26T21:10:03.707438+0000","last_undegraded":"2024-04-26T21:10:03.707438+0000","last_fullsized":"2024-04-26T21:10:03.707438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:38.761142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698273+0000","last_change":"2024-04-26T21:09:40.387004+0000","last_active":"2024-04-26T21:10:03.698273+0000","last_peered":"2024-04-26T21:10:03.698273+0000","last_clean":"2024-04-26T21:10:03.698273+0000","last_became_active":"2024-04-26T21:09:40.382385+0000","last_became_peered":"2024-04-26T21:09:40.382385+0000","last_unstale":"2024-04-26T21:10:03.698273+0000","last_undegraded":"2024-04-26T21:10:03.698273+0000","last_fullsized":"2024-04-26T21:10:03.698273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:40.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.696898+0000","last_change":"2024-04-26T21:09:41.071024+0000","last_active":"2024-04-26T21:10:03.696898+0000","last_peered":"2024-04-26T21:10:03.696898+0000","last_clean":"2024-04-26T21:10:03.696898+0000","last_became_active":"2024-04-26T21:09:41.070456+0000","last_became_peered":"2024-04-26T21:09:41.070456+0000","last_unstale":"2024-04-26T21:10:03.696898+0000","last_undegraded":"2024-04-26T21:10:03.696898+0000","last_fullsized":"2024-04-26T21:10:03.696898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:20:56.046786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701415+0000","last_change":"2024-04-26T21:09:40.404128+0000","last_active":"2024-04-26T21:10:03.701415+0000","last_peered":"2024-04-26T21:10:03.701415+0000","last_clean":"2024-04-26T21:10:03.701415+0000","last_became_active":"2024-04-26T21:09:40.403760+0000","last_became_peered":"2024-04-26T21:09:40.403760+0000","last_unstale":"2024-04-26T21:10:03.701415+0000","last_undegraded":"2024-04-26T21:10:03.701415+0000","last_fullsized":"2024-04-26T21:10:03.701415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698992+0000","last_change":"2024-04-26T21:09:41.067676+0000","last_active":"2024-04-26T21:10:03.698992+0000","last_peered":"2024-04-26T21:10:03.698992+0000","last_clean":"2024-04-26T21:10:03.698992+0000","last_became_active":"2024-04-26T21:09:41.066736+0000","last_became_peered":"2024-04-26T21:09:41.066736+0000","last_unstale":"2024-04-26T21:10:03.698992+0000","last_undegraded":"2024-04-26T21:10:03.698992+0000","last_fullsized":"2024-04-26T21:10:03.698992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:39.477690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.724113+0000","last_change":"2024-04-26T21:10:03.722877+0000","last_active":"2024-04-26T21:10:03.724113+0000","last_peered":"2024-04-26T21:10:03.724113+0000","last_clean":"2024-04-26T21:10:03.724113+0000","last_became_active":"2024-04-26T21:10:03.710433+0000","last_became_peered":"2024-04-26T21:10:03.710433+0000","last_unstale":"2024-04-26T21:10:03.724113+0000","last_undegraded":"2024-04-26T21:10:03.724113+0000","last_fullsized":"2024-04-26T21:10:03.724113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:59.042373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691088+0000","last_change":"2024-04-26T21:09:41.074270+0000","last_active":"2024-04-26T21:10:03.691088+0000","last_peered":"2024-04-26T21:10:03.691088+0000","last_clean":"2024-04-26T21:10:03.691088+0000","last_became_active":"2024-04-26T21:09:41.073847+0000","last_became_peered":"2024-04-26T21:09:41.073847+0000","last_unstale":"2024-04-26T21:10:03.691088+0000","last_undegraded":"2024-04-26T21:10:03.691088+0000","last_fullsized":"2024-04-26T21:10:03.691088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:03.737883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:11:10.730594+0000","last_change":"2024-04-26T21:09:40.397644+0000","last_active":"2024-04-26T21:11:10.730594+0000","last_peered":"2024-04-26T21:11:10.730594+0000","last_clean":"2024-04-26T21:11:10.730594+0000","last_became_active":"2024-04-26T21:09:40.397234+0000","last_became_peered":"2024-04-26T21:09:40.397234+0000","last_unstale":"2024-04-26T21:11:10.730594+0000","last_undegraded":"2024-04-26T21:11:10.730594+0000","last_fullsized":"2024-04-26T21:11:10.730594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:32.220330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691262+0000","last_change":"2024-04-26T21:09:41.072171+0000","last_active":"2024-04-26T21:10:03.691262+0000","last_peered":"2024-04-26T21:10:03.691262+0000","last_clean":"2024-04-26T21:10:03.691262+0000","last_became_active":"2024-04-26T21:09:41.071706+0000","last_became_peered":"2024-04-26T21:09:41.071706+0000","last_unstale":"2024-04-26T21:10:03.691262+0000","last_undegraded":"2024-04-26T21:10:03.691262+0000","last_fullsized":"2024-04-26T21:10:03.691262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:14.286865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691309+0000","last_change":"2024-04-26T21:09:40.393724+0000","last_active":"2024-04-26T21:10:03.691309+0000","last_peered":"2024-04-26T21:10:03.691309+0000","last_clean":"2024-04-26T21:10:03.691309+0000","last_became_active":"2024-04-26T21:09:40.392441+0000","last_became_peered":"2024-04-26T21:09:40.392441+0000","last_unstale":"2024-04-26T21:10:03.691309+0000","last_undegraded":"2024-04-26T21:10:03.691309+0000","last_fullsized":"2024-04-26T21:10:03.691309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:11.989418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.717632+0000","last_change":"2024-04-26T21:10:03.717632+0000","last_active":"2024-04-26T21:10:03.717632+0000","last_peered":"2024-04-26T21:10:03.717632+0000","last_clean":"2024-04-26T21:10:03.717632+0000","last_became_active":"2024-04-26T21:10:03.705478+0000","last_became_peered":"2024-04-26T21:10:03.705478+0000","last_unstale":"2024-04-26T21:10:03.717632+0000","last_undegraded":"2024-04-26T21:10:03.717632+0000","last_fullsized":"2024-04-26T21:10:03.717632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.259935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691920+0000","last_change":"2024-04-26T21:09:40.400884+0000","last_active":"2024-04-26T21:10:03.691920+0000","last_peered":"2024-04-26T21:10:03.691920+0000","last_clean":"2024-04-26T21:10:03.691920+0000","last_became_active":"2024-04-26T21:09:40.400326+0000","last_became_peered":"2024-04-26T21:09:40.400326+0000","last_unstale":"2024-04-26T21:10:03.691920+0000","last_undegraded":"2024-04-26T21:10:03.691920+0000","last_fullsized":"2024-04-26T21:10:03.691920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:39.341776+0000","last_clean_scrub_stamp":"2024-04-26T21:09:39.341776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:13.114233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691193+0000","last_change":"2024-04-26T21:09:41.071959+0000","last_active":"2024-04-26T21:10:03.691193+0000","last_peered":"2024-04-26T21:10:03.691193+0000","last_clean":"2024-04-26T21:10:03.691193+0000","last_became_active":"2024-04-26T21:09:41.071601+0000","last_became_peered":"2024-04-26T21:09:41.071601+0000","last_unstale":"2024-04-26T21:10:03.691193+0000","last_undegraded":"2024-04-26T21:10:03.691193+0000","last_fullsized":"2024-04-26T21:10:03.691193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:29.806956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697259+0000","last_change":"2024-04-26T21:09:41.066764+0000","last_active":"2024-04-26T21:10:03.697259+0000","last_peered":"2024-04-26T21:10:03.697259+0000","last_clean":"2024-04-26T21:10:03.697259+0000","last_became_active":"2024-04-26T21:09:41.066333+0000","last_became_peered":"2024-04-26T21:09:41.066333+0000","last_unstale":"2024-04-26T21:10:03.697259+0000","last_undegraded":"2024-04-26T21:10:03.697259+0000","last_fullsized":"2024-04-26T21:10:03.697259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:32.680215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.700693+0000","last_change":"2024-04-26T21:09:41.072567+0000","last_active":"2024-04-26T21:10:03.700693+0000","last_peered":"2024-04-26T21:10:03.700693+0000","last_clean":"2024-04-26T21:10:03.700693+0000","last_became_active":"2024-04-26T21:09:41.072033+0000","last_became_peered":"2024-04-26T21:09:41.072033+0000","last_unstale":"2024-04-26T21:10:03.700693+0000","last_undegraded":"2024-04-26T21:10:03.700693+0000","last_fullsized":"2024-04-26T21:10:03.700693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:37.178970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.701116+0000","last_change":"2024-04-26T21:09:41.071111+0000","last_active":"2024-04-26T21:10:03.701116+0000","last_peered":"2024-04-26T21:10:03.701116+0000","last_clean":"2024-04-26T21:10:03.701116+0000","last_became_active":"2024-04-26T21:09:41.070354+0000","last_became_peered":"2024-04-26T21:09:41.070354+0000","last_unstale":"2024-04-26T21:10:03.701116+0000","last_undegraded":"2024-04-26T21:10:03.701116+0000","last_fullsized":"2024-04-26T21:10:03.701116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:10.990058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.716445+0000","last_change":"2024-04-26T21:09:41.076927+0000","last_active":"2024-04-26T21:10:03.716445+0000","last_peered":"2024-04-26T21:10:03.716445+0000","last_clean":"2024-04-26T21:10:03.716445+0000","last_became_active":"2024-04-26T21:09:41.076600+0000","last_became_peered":"2024-04-26T21:09:41.076600+0000","last_unstale":"2024-04-26T21:10:03.716445+0000","last_undegraded":"2024-04-26T21:10:03.716445+0000","last_fullsized":"2024-04-26T21:10:03.716445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:25.960409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691916+0000","last_change":"2024-04-26T21:09:41.079293+0000","last_active":"2024-04-26T21:10:03.691916+0000","last_peered":"2024-04-26T21:10:03.691916+0000","last_clean":"2024-04-26T21:10:03.691916+0000","last_became_active":"2024-04-26T21:09:41.077473+0000","last_became_peered":"2024-04-26T21:09:41.077473+0000","last_unstale":"2024-04-26T21:10:03.691916+0000","last_undegraded":"2024-04-26T21:10:03.691916+0000","last_fullsized":"2024-04-26T21:10:03.691916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:47.281535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697787+0000","last_change":"2024-04-26T21:09:41.065914+0000","last_active":"2024-04-26T21:10:03.697787+0000","last_peered":"2024-04-26T21:10:03.697787+0000","last_clean":"2024-04-26T21:10:03.697787+0000","last_became_active":"2024-04-26T21:09:41.065477+0000","last_became_peered":"2024-04-26T21:09:41.065477+0000","last_unstale":"2024-04-26T21:10:03.697787+0000","last_undegraded":"2024-04-26T21:10:03.697787+0000","last_fullsized":"2024-04-26T21:10:03.697787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:40.517174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.704127+0000","last_change":"2024-04-26T21:09:41.068594+0000","last_active":"2024-04-26T21:10:03.704127+0000","last_peered":"2024-04-26T21:10:03.704127+0000","last_clean":"2024-04-26T21:10:03.704127+0000","last_became_active":"2024-04-26T21:09:41.068238+0000","last_became_peered":"2024-04-26T21:09:41.068238+0000","last_unstale":"2024-04-26T21:10:03.704127+0000","last_undegraded":"2024-04-26T21:10:03.704127+0000","last_fullsized":"2024-04-26T21:10:03.704127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:58.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.699267+0000","last_change":"2024-04-26T21:09:41.081684+0000","last_active":"2024-04-26T21:10:03.699267+0000","last_peered":"2024-04-26T21:10:03.699267+0000","last_clean":"2024-04-26T21:10:03.699267+0000","last_became_active":"2024-04-26T21:09:41.074963+0000","last_became_peered":"2024-04-26T21:09:41.074963+0000","last_unstale":"2024-04-26T21:10:03.699267+0000","last_undegraded":"2024-04-26T21:10:03.699267+0000","last_fullsized":"2024-04-26T21:10:03.699267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:14.736390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.691381+0000","last_change":"2024-04-26T21:09:41.068052+0000","last_active":"2024-04-26T21:10:03.691381+0000","last_peered":"2024-04-26T21:10:03.691381+0000","last_clean":"2024-04-26T21:10:03.691381+0000","last_became_active":"2024-04-26T21:09:41.067728+0000","last_became_peered":"2024-04-26T21:09:41.067728+0000","last_unstale":"2024-04-26T21:10:03.691381+0000","last_undegraded":"2024-04-26T21:10:03.691381+0000","last_fullsized":"2024-04-26T21:10:03.691381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:19.186963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.698395+0000","last_change":"2024-04-26T21:09:41.078725+0000","last_active":"2024-04-26T21:10:03.698395+0000","last_peered":"2024-04-26T21:10:03.698395+0000","last_clean":"2024-04-26T21:10:03.698395+0000","last_became_active":"2024-04-26T21:09:41.075757+0000","last_became_peered":"2024-04-26T21:09:41.075757+0000","last_unstale":"2024-04-26T21:10:03.698395+0000","last_undegraded":"2024-04-26T21:10:03.698395+0000","last_fullsized":"2024-04-26T21:10:03.698395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:50.280489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697276+0000","last_change":"2024-04-26T21:09:41.067872+0000","last_active":"2024-04-26T21:10:03.697276+0000","last_peered":"2024-04-26T21:10:03.697276+0000","last_clean":"2024-04-26T21:10:03.697276+0000","last_became_active":"2024-04-26T21:09:41.067476+0000","last_became_peered":"2024-04-26T21:09:41.067476+0000","last_unstale":"2024-04-26T21:10:03.697276+0000","last_undegraded":"2024-04-26T21:10:03.697276+0000","last_fullsized":"2024-04-26T21:10:03.697276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:54.740719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.692241+0000","last_change":"2024-04-26T21:09:41.080571+0000","last_active":"2024-04-26T21:10:03.692241+0000","last_peered":"2024-04-26T21:10:03.692241+0000","last_clean":"2024-04-26T21:10:03.692241+0000","last_became_active":"2024-04-26T21:09:41.078465+0000","last_became_peered":"2024-04-26T21:09:41.078465+0000","last_unstale":"2024-04-26T21:10:03.692241+0000","last_undegraded":"2024-04-26T21:10:03.692241+0000","last_fullsized":"2024-04-26T21:10:03.692241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:27.636693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:10:03.697764+0000","last_change":"2024-04-26T21:09:41.080111+0000","last_active":"2024-04-26T21:10:03.697764+0000","last_peered":"2024-04-26T21:10:03.697764+0000","last_clean":"2024-04-26T21:10:03.697764+0000","last_became_active":"2024-04-26T21:09:41.079220+0000","last_became_peered":"2024-04-26T21:09:41.079220+0000","last_unstale":"2024-04-26T21:10:03.697764+0000","last_undegraded":"2024-04-26T21:10:03.697764+0000","last_fullsized":"2024-04-26T21:10:03.697764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:40.353124+0000","last_clean_scrub_stamp":"2024-04-26T21:09:40.353124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.055957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60195,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":853,"num_read_kb":485,"num_write":238,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":48838,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":180585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285604,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":471504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1115634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":470879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972409,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":470468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1062986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":470479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":525446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":471818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":471612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":470187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":524113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1622016,"data_stored":1062945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":55806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:11:18.407 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T21:11:18.407 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:11:18.407 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:11:18.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T21:11:18.926 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:18.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:18 smithi027 ceph-mon[28085]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:19.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:18 smithi204 ceph-mon[32145]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:19.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:18 smithi094 ceph-mon[31273]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:11:19.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:19 smithi027 ceph-mon[28085]: pgmap v418: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:19 smithi204 ceph-mon[32145]: pgmap v418: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:20.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:19 smithi094 ceph-mon[31273]: pgmap v418: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.050+0000 7ffb74200700 1 -- 172.21.15.27:0/93195353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c06c4a0 msgr2=0x7ffb6c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.050+0000 7ffb74200700 1 --2- 172.21.15.27:0/93195353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1102e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400b0a0 tx=0x7ffb6400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 -- 172.21.15.27:0/93195353 shutdown_connections 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 --2- 172.21.15.27:0/93195353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1102e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 --2- 172.21.15.27:0/93195353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 --2- 172.21.15.27:0/93195353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 0x7ffb6c06bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 -- 172.21.15.27:0/93195353 >> 172.21.15.27:0/93195353 conn(0x7ffb6c079290 msgr2=0x7ffb6c0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 -- 172.21.15.27:0/93195353 shutdown_connections 2024-04-26T21:11:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.051+0000 7ffb74200700 1 -- 172.21.15.27:0/93195353 wait complete. 2024-04-26T21:11:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.052+0000 7ffb74200700 1 Processor -- start 2024-04-26T21:11:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.052+0000 7ffb74200700 1 -- start start 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.052+0000 7ffb74200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 0x7ffb6c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb74200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb74200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb74200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c1134a0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb74200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c113320 con 0x7ffb6c06bb00 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb74200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c113620 con 0x7ffb6c06c4a0 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51956/0 (socket says 172.21.15.27:51956) 2024-04-26T21:11:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 -- 172.21.15.27:0/2093881469 learned_addr learned my addr 172.21.15.27:0/2093881469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb70e00700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb71800700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 0x7ffb6c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 -- 172.21.15.27:0/2093881469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c06c4a0 msgr2=0x7ffb6c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 -- 172.21.15.27:0/2093881469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 msgr2=0x7ffb6c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 0x7ffb6c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.053+0000 7ffb72200700 1 -- 172.21.15.27:0/2093881469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c19c220 con 0x7ffb6c1094a0 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb72200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400b070 tx=0x7ffb64004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64021070 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb64007c00 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c19c4b0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64004780 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.054+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c1a87d0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.056+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c06b3a0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.058+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb64003f20 con 0x7ffb6c1094a0 2024-04-26T21:11:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.059+0000 7ffb6a000700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 0x7ffb540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.059+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ffb6400f020 con 0x7ffb6c1094a0 2024-04-26T21:11:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.060+0000 7ffb71800700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 0x7ffb540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.060+0000 7ffb71800700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 0x7ffb540a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c19cc00 tx=0x7ffb6000b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.062+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffb6402a020 con 0x7ffb6c1094a0 2024-04-26T21:11:21.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.347+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffb6c19cff0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.348+0000 7ffb6a000700 1 -- 172.21.15.27:0/2093881469 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ffb6409e7e0 con 0x7ffb6c1094a0 2024-04-26T21:11:21.350 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:21.350 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:11:21.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.350+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 msgr2=0x7ffb540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:21.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.350+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 0x7ffb540a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c19cc00 tx=0x7ffb6000b4d0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.350+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 msgr2=0x7ffb6c19bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.350+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400b070 tx=0x7ffb64004e00 comp rx=0 tx=0).stop 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 shutdown_connections 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ffb540a3cd0 0x7ffb540a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb6c1094a0 0x7ffb6c19bac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb6c06bb00 0x7ffb6c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 --2- 172.21.15.27:0/2093881469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb6c06c4a0 0x7ffb6c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:21.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 >> 172.21.15.27:0/2093881469 conn(0x7ffb6c079290 msgr2=0x7ffb6c10e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:21.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 shutdown_connections 2024-04-26T21:11:21.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:21.351+0000 7ffb74200700 1 -- 172.21.15.27:0/2093881469 wait complete. 2024-04-26T21:11:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:21 smithi027 ceph-mon[28085]: pgmap v419: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:21.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:21 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2093881469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:11:22.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:21 smithi204 ceph-mon[32145]: pgmap v419: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:22.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:21 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2093881469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:11:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:21 smithi094 ceph-mon[31273]: pgmap v419: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:21 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2093881469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:11:22.182 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:11:22.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:22.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 -- 172.21.15.27:0/3218807245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4076910 msgr2=0x7fb8a4074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 --2- 172.21.15.27:0/3218807245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4076910 0x7fb8a4074d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb898009900 tx=0x7fb89802a480 comp rx=0 tx=0).stop 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 -- 172.21.15.27:0/3218807245 shutdown_connections 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 --2- 172.21.15.27:0/3218807245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a4075330 0x7fb8a4075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 --2- 172.21.15.27:0/3218807245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a410a800 0x7fb8a410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 --2- 172.21.15.27:0/3218807245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4076910 0x7fb8a4074d60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 -- 172.21.15.27:0/3218807245 >> 172.21.15.27:0/3218807245 conn(0x7fb8a40fdee0 msgr2=0x7fb8a4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 -- 172.21.15.27:0/3218807245 shutdown_connections 2024-04-26T21:11:22.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.403+0000 7fb8ac200640 1 -- 172.21.15.27:0/3218807245 wait complete. 2024-04-26T21:11:22.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.404+0000 7fb8ac200640 1 Processor -- start 2024-04-26T21:11:22.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.404+0000 7fb8ac200640 1 -- start start 2024-04-26T21:11:22.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a4076910 0x7fb8a419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a410a800 0x7fb8a41a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4110590 con 0x7fb8a4075330 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4110410 con 0x7fb8a4076910 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8ac200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4110710 con 0x7fb8a410a800 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51974/0 (socket says 172.21.15.27:51974) 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 -- 172.21.15.27:0/617870760 learned_addr learned my addr 172.21.15.27:0/617870760 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8aa200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a410a800 0x7fb8a41a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a8e00640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a4076910 0x7fb8a419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 -- 172.21.15.27:0/617870760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a410a800 msgr2=0x7fb8a41a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a410a800 0x7fb8a41a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 -- 172.21.15.27:0/617870760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a4076910 msgr2=0x7fb8a419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a4076910 0x7fb8a419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.405+0000 7fb8a9800640 1 -- 172.21.15.27:0/617870760 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a41a28f0 con 0x7fb8a4075330 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8a9800640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb898031460 tx=0x7fb898003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89803f070 con 0x7fb8a4075330 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb898031da0 con 0x7fb8a4075330 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898035030 con 0x7fb8a4075330 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a41a2b80 con 0x7fb8a4075330 2024-04-26T21:11:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.406+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41a3020 con 0x7fb8a4075330 2024-04-26T21:11:22.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.408+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a41095c0 con 0x7fb8a4075330 2024-04-26T21:11:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.410+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb898042070 con 0x7fb8a4075330 2024-04-26T21:11:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.411+0000 7fb8a2000640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 0x7fb88c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.411+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb8980f0f50 con 0x7fb8a4075330 2024-04-26T21:11:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.411+0000 7fb8a8e00640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 0x7fb88c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.412+0000 7fb8a8e00640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 0x7fb88c0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb890005fd0 tx=0x7fb890005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.415+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb8980bd760 con 0x7fb8a4075330 2024-04-26T21:11:22.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.623+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8a402dd40 con 0x7fb8a4075330 2024-04-26T21:11:22.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.626+0000 7fb8a2000640 1 -- 172.21.15.27:0/617870760 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7fb8980bd060 con 0x7fb8a4075330 2024-04-26T21:11:22.628 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:22.628 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:22.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 msgr2=0x7fb88c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 0x7fb88c0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb890005fd0 tx=0x7fb890005850 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 msgr2=0x7fb8a419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb898031460 tx=0x7fb898003c80 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 shutdown_connections 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb88c0a3cc0 0x7fb88c0a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8a410a800 0x7fb8a41a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8a4076910 0x7fb8a419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 --2- 172.21.15.27:0/617870760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a4075330 0x7fb8a419acb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.629+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 >> 172.21.15.27:0/617870760 conn(0x7fb8a40fdee0 msgr2=0x7fb8a4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.630+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 shutdown_connections 2024-04-26T21:11:22.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.630+0000 7fb8ac200640 1 -- 172.21.15.27:0/617870760 wait complete. 2024-04-26T21:11:22.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:22.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 -- 172.21.15.27:0/516370484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff758105c90 msgr2=0x7ff758108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 --2- 172.21.15.27:0/516370484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff758105c90 0x7ff758108080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff740009900 tx=0x7ff74002a410 comp rx=0 tx=0).stop 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 -- 172.21.15.27:0/516370484 shutdown_connections 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 --2- 172.21.15.27:0/516370484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7581085c0 0x7ff75810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 --2- 172.21.15.27:0/516370484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff758105c90 0x7ff758108080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 --2- 172.21.15.27:0/516370484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 0x7ff758105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.848+0000 7ff75ca00640 1 -- 172.21.15.27:0/516370484 >> 172.21.15.27:0/516370484 conn(0x7ff7580fccd0 msgr2=0x7ff7580ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.849+0000 7ff75ca00640 1 -- 172.21.15.27:0/516370484 shutdown_connections 2024-04-26T21:11:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.849+0000 7ff75ca00640 1 -- 172.21.15.27:0/516370484 wait complete. 2024-04-26T21:11:22.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.849+0000 7ff75ca00640 1 Processor -- start 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 -- start start 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 0x7ff75819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff758105c90 0x7ff75819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810e260 con 0x7ff7581085c0 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810e0e0 con 0x7ff758105c90 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff75ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810e3e0 con 0x7ff758103360 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff757200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff757200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51984/0 (socket says 172.21.15.27:51984) 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff757200640 1 -- 172.21.15.27:0/423534966 learned_addr learned my addr 172.21.15.27:0/423534966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff74de00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff758105c90 0x7ff75819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.850+0000 7ff756800640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 0x7ff75819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 -- 172.21.15.27:0/423534966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 msgr2=0x7ff75819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 0x7ff75819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 -- 172.21.15.27:0/423534966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff758105c90 msgr2=0x7ff75819b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff758105c90 0x7ff75819b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 -- 172.21.15.27:0/423534966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581a27e0 con 0x7ff7581085c0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff757200640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff75000cad0 tx=0x7ff75000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75000ae20 con 0x7ff7581085c0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff750005020 con 0x7ff7581085c0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581a2ad0 con 0x7ff7581085c0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75000f6b0 con 0x7ff7581085c0 2024-04-26T21:11:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.851+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a2fa0 con 0x7ff7581085c0 2024-04-26T21:11:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.852+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75810a0b0 con 0x7ff7581085c0 2024-04-26T21:11:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.854+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff7500051d0 con 0x7ff7581085c0 2024-04-26T21:11:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.856+0000 7ff74fe00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 0x7ff73c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.856+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff7500d0a50 con 0x7ff7581085c0 2024-04-26T21:11:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.856+0000 7ff756800640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 0x7ff73c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.857+0000 7ff756800640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 0x7ff73c0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff748005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:22.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:22.860+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff750014030 con 0x7ff7581085c0 2024-04-26T21:11:22.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:22 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/617870760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:23.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:22 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/617870760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:23.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:22 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/617870760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.058+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff75802dcd0 con 0x7ff7581085c0 2024-04-26T21:11:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.059+0000 7ff74fe00640 1 -- 172.21.15.27:0/423534966 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff75009d260 con 0x7ff7581085c0 2024-04-26T21:11:23.060 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.061+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 msgr2=0x7ff73c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.061+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 0x7ff73c0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff748005e20 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.061+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 msgr2=0x7ff7581a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.061+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff75000cad0 tx=0x7ff75000cf90 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 shutdown_connections 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff73c0a3c40 0x7ff73c0a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7581085c0 0x7ff7581a2170 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff758105c90 0x7ff75819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 --2- 172.21.15.27:0/423534966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff758103360 0x7ff75819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 >> 172.21.15.27:0/423534966 conn(0x7ff7580fccd0 msgr2=0x7ff758102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 shutdown_connections 2024-04-26T21:11:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.062+0000 7ff75ca00640 1 -- 172.21.15.27:0/423534966 wait complete. 2024-04-26T21:11:23.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:11:23.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 -- 172.21.15.27:0/2581447157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 msgr2=0x7f4de4075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 --2- 172.21.15.27:0/2581447157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de4075790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009ca0 tx=0x7f4dcc02a950 comp rx=0 tx=0).stop 2024-04-26T21:11:23.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 -- 172.21.15.27:0/2581447157 shutdown_connections 2024-04-26T21:11:23.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 --2- 172.21.15.27:0/2581447157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 0x7f4de410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 --2- 172.21.15.27:0/2581447157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de4075790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 --2- 172.21.15.27:0/2581447157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 0x7f4de4074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 -- 172.21.15.27:0/2581447157 >> 172.21.15.27:0/2581447157 conn(0x7f4de40fdee0 msgr2=0x7f4de4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:23.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 -- 172.21.15.27:0/2581447157 shutdown_connections 2024-04-26T21:11:23.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.300+0000 7f4deb800640 1 -- 172.21.15.27:0/2581447157 wait complete. 2024-04-26T21:11:23.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.301+0000 7f4deb800640 1 Processor -- start 2024-04-26T21:11:23.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.301+0000 7f4deb800640 1 -- start start 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.301+0000 7f4deb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.301+0000 7f4deb800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 0x7f4de419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4deb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 0x7f4de41a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4deb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4110530 con 0x7f4de4075330 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4deb800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41103b0 con 0x7f4de410a800 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4deb800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41106b0 con 0x7f4de4076910 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51998/0 (socket says 172.21.15.27:51998) 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 -- 172.21.15.27:0/2810186699 learned_addr learned my addr 172.21.15.27:0/2810186699 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de3e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 0x7f4de419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de9800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 0x7f4de41a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 -- 172.21.15.27:0/2810186699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 msgr2=0x7f4de419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 0x7f4de419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 -- 172.21.15.27:0/2810186699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 msgr2=0x7f4de41a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 0x7f4de41a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.302+0000 7f4de8e00640 1 -- 172.21.15.27:0/2810186699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41a2940 con 0x7f4de4075330 2024-04-26T21:11:23.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4de8e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400d950 tx=0x7f4dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4017070 con 0x7f4de4075330 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dd4009c30 con 0x7f4de4075330 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de41a2c30 con 0x7f4de4075330 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400f620 con 0x7f4de4075330 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.303+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41a3180 con 0x7f4de4075330 2024-04-26T21:11:23.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.304+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de41095c0 con 0x7f4de4075330 2024-04-26T21:11:23.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.307+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4dd4010460 con 0x7f4de4075330 2024-04-26T21:11:23.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.308+0000 7f4de1600640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 0x7f4dc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.308+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4dd40d0350 con 0x7f4de4075330 2024-04-26T21:11:23.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.308+0000 7f4de3e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 0x7f4dc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.309+0000 7f4de3e00640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 0x7f4dc40a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc02f8c0 tx=0x7f4dcc009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:23.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.313+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4dd409cb60 con 0x7f4de4075330 2024-04-26T21:11:23.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.525+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4de4063530 con 0x7f4de4075330 2024-04-26T21:11:23.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.525+0000 7f4de1600640 1 -- 172.21.15.27:0/2810186699 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f4dd4009de0 con 0x7f4de4075330 2024-04-26T21:11:23.527 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:11:23.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 msgr2=0x7f4dc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 0x7f4dc40a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc02f8c0 tx=0x7f4dcc009360 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 msgr2=0x7f4de419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400d950 tx=0x7f4dd400de10 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 shutdown_connections 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4dc40a3c40 0x7f4dc40a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de4076910 0x7f4de419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4de410a800 0x7f4de41a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 --2- 172.21.15.27:0/2810186699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de4075330 0x7f4de419ad30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.528+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 >> 172.21.15.27:0/2810186699 conn(0x7f4de40fdee0 msgr2=0x7f4de4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.529+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 shutdown_connections 2024-04-26T21:11:23.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.529+0000 7f4deb800640 1 -- 172.21.15.27:0/2810186699 wait complete. 2024-04-26T21:11:23.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 -- 172.21.15.27:0/1659753767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 msgr2=0x7f50f4102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 --2- 172.21.15.27:0/1659753767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f4102ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009900 tx=0x7f50e402a400 comp rx=0 tx=0).stop 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 -- 172.21.15.27:0/1659753767 shutdown_connections 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 --2- 172.21.15.27:0/1659753767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4104330 0x7f50f410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 --2- 172.21.15.27:0/1659753767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4103970 0x7f50f4103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 --2- 172.21.15.27:0/1659753767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f4102ba0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.775+0000 7f50f8a00640 1 -- 172.21.15.27:0/1659753767 >> 172.21.15.27:0/1659753767 conn(0x7f50f40fdcf0 msgr2=0x7f50f4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.776+0000 7f50f8a00640 1 -- 172.21.15.27:0/1659753767 shutdown_connections 2024-04-26T21:11:23.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.776+0000 7f50f8a00640 1 -- 172.21.15.27:0/1659753767 wait complete. 2024-04-26T21:11:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 Processor -- start 2024-04-26T21:11:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 -- start start 2024-04-26T21:11:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 0x7f50f419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4104330 0x7f50f41a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410e200 con 0x7f50f4102780 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410e080 con 0x7f50f4104330 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410e380 con 0x7f50f4103970 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52012/0 (socket says 172.21.15.27:52012) 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.777+0000 7f50f2800640 1 -- 172.21.15.27:0/644203441 learned_addr learned my addr 172.21.15.27:0/644203441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f3200640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4104330 0x7f50f41a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f1e00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 0x7f50f419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 -- 172.21.15.27:0/644203441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 msgr2=0x7f50f419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 0x7f50f419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 -- 172.21.15.27:0/644203441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4104330 msgr2=0x7f50f41a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4104330 0x7f50f41a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 -- 172.21.15.27:0/644203441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a26d0 con 0x7f50f4102780 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f1e00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 0x7f50f419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50f2800640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50e4031460 tx=0x7f50e4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e403f070 con 0x7f50f4102780 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.778+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e4031da0 con 0x7f50f4102780 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.779+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a2960 con 0x7f50f4102780 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.779+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4035030 con 0x7f50f4102780 2024-04-26T21:11:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.779+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a2ed0 con 0x7f50f4102780 2024-04-26T21:11:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.780+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f41a30a0 con 0x7f50f4102780 2024-04-26T21:11:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.782+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50e4042070 con 0x7f50f4102780 2024-04-26T21:11:23.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.783+0000 7f50eb400640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 0x7f50d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:23.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.783+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f50e40f1010 con 0x7f50f4102780 2024-04-26T21:11:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.784+0000 7f50f1e00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 0x7f50d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.784+0000 7f50f1e00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 0x7f50d40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50f419bfc0 tx=0x7f50dc008350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:23.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.788+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50e40bd880 con 0x7f50f4102780 2024-04-26T21:11:23.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:23 smithi027 ceph-mon[28085]: pgmap v420: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:23.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:23 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/423534966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:23.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:23 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2810186699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:23.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.994+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50f41a3270 con 0x7f50f4102780 2024-04-26T21:11:23.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.995+0000 7f50eb400640 1 -- 172.21.15.27:0/644203441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f50e40bd180 con 0x7f50f4102780 2024-04-26T21:11:23.996 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 msgr2=0x7f50d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 0x7f50d40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50f419bfc0 tx=0x7f50dc008350 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 msgr2=0x7f50f419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50e4031460 tx=0x7f50e4003d50 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 shutdown_connections 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f50d40a3c40 0x7f50d40a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f4103970 0x7f50f419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f4104330 0x7f50f41a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 --2- 172.21.15.27:0/644203441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50f4102780 0x7f50f419aab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.997+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 >> 172.21.15.27:0/644203441 conn(0x7f50f40fdcf0 msgr2=0x7f50f4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.998+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 shutdown_connections 2024-04-26T21:11:23.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:23.998+0000 7f50f8a00640 1 -- 172.21.15.27:0/644203441 wait complete. 2024-04-26T21:11:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:23 smithi204 ceph-mon[32145]: pgmap v420: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:23 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/423534966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:24.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:23 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2810186699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:24.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:24.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:23 smithi094 ceph-mon[31273]: pgmap v420: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:24.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:23 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/423534966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:24.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:23 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2810186699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 -- 172.21.15.27:0/4069611330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 msgr2=0x7f9c7010a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 --2- 172.21.15.27:0/4069611330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7010a8a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009ca0 tx=0x7f9c6802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 -- 172.21.15.27:0/4069611330 shutdown_connections 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 --2- 172.21.15.27:0/4069611330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7010a8a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 --2- 172.21.15.27:0/4069611330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c70105b20 0x7f9c70107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 --2- 172.21.15.27:0/4069611330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c701031f0 0x7f9c701055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 -- 172.21.15.27:0/4069611330 >> 172.21.15.27:0/4069611330 conn(0x7f9c700fcbc0 msgr2=0x7f9c700ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 -- 172.21.15.27:0/4069611330 shutdown_connections 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.238+0000 7f9c74c00640 1 -- 172.21.15.27:0/4069611330 wait complete. 2024-04-26T21:11:24.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.239+0000 7f9c74c00640 1 Processor -- start 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 -- start start 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 0x7f9c70198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70105b20 0x7f9c70198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010e1a0 con 0x7f9c70108450 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010e020 con 0x7f9c701031f0 2024-04-26T21:11:24.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c74c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7010e320 con 0x7f9c70105b20 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52022/0 (socket says 172.21.15.27:52022) 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c6fe00640 1 -- 172.21.15.27:0/1984211412 learned_addr learned my addr 172.21.15.27:0/1984211412 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c6f400640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 0x7f9c70198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.240+0000 7f9c67e00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70105b20 0x7f9c70198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 -- 172.21.15.27:0/1984211412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70105b20 msgr2=0x7f9c70198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70105b20 0x7f9c70198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 -- 172.21.15.27:0/1984211412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 msgr2=0x7f9c70198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 0x7f9c70198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 -- 172.21.15.27:0/1984211412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c701a0460 con 0x7f9c70108450 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6f400640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 0x7f9c70198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6fe00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802aeb0 tx=0x7f9c68003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68032ed0 con 0x7f9c70108450 2024-04-26T21:11:24.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c68003e10 con 0x7f9c70108450 2024-04-26T21:11:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c701a06f0 con 0x7f9c70108450 2024-04-26T21:11:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.241+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68030cd0 con 0x7f9c70108450 2024-04-26T21:11:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.242+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c701a0c90 con 0x7f9c70108450 2024-04-26T21:11:24.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.243+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c700fe6b0 con 0x7f9c70108450 2024-04-26T21:11:24.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.245+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c68004b40 con 0x7f9c70108450 2024-04-26T21:11:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.246+0000 7f9c6cc00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 0x7f9c540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.247+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9c680f1d50 con 0x7f9c70108450 2024-04-26T21:11:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.247+0000 7f9c6f400640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 0x7f9c540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.247+0000 7f9c6f400640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 0x7f9c540a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c600058c0 tx=0x7f9c60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:24.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.249+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9c680be670 con 0x7f9c70108450 2024-04-26T21:11:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.509+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c7002dad0 con 0x7f9c70108450 2024-04-26T21:11:24.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.511+0000 7f9c6cc00640 1 -- 172.21.15.27:0/1984211412 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7f9c680bdf70 con 0x7f9c70108450 2024-04-26T21:11:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:24.513 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:11:24.513 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.514+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 msgr2=0x7f9c540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.514+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 0x7f9c540a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c600058c0 tx=0x7f9c60005850 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.514+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 msgr2=0x7f9c7019fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.514+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802aeb0 tx=0x7f9c68003c20 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 shutdown_connections 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f9c540a3cc0 0x7f9c540a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c70105b20 0x7f9c70198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c70108450 0x7f9c7019fdf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 --2- 172.21.15.27:0/1984211412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c701031f0 0x7f9c70198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 >> 172.21.15.27:0/1984211412 conn(0x7f9c700fcbc0 msgr2=0x7f9c70103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 shutdown_connections 2024-04-26T21:11:24.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.515+0000 7f9c74c00640 1 -- 172.21.15.27:0/1984211412 wait complete. 2024-04-26T21:11:24.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 -- 172.21.15.27:0/3007975929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10101780 msgr2=0x7fef10101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3007975929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10101780 0x7fef10101c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feef8009900 tx=0x7feef802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 -- 172.21.15.27:0/3007975929 shutdown_connections 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3007975929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10102140 0x7fef101029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3007975929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10101780 0x7fef10101c00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3007975929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef10100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.755+0000 7fef0fe00640 1 -- 172.21.15.27:0/3007975929 >> 172.21.15.27:0/3007975929 conn(0x7fef100fc390 msgr2=0x7fef100fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.756+0000 7fef0fe00640 1 -- 172.21.15.27:0/3007975929 shutdown_connections 2024-04-26T21:11:24.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.756+0000 7fef0fe00640 1 -- 172.21.15.27:0/3007975929 wait complete. 2024-04-26T21:11:24.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 Processor -- start 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 -- start start 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef1019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10101780 0x7fef1019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef10110340 con 0x7fef10102140 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef101101c0 con 0x7fef10100a20 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef101104c0 con 0x7fef10101780 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.757+0000 7fef0f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52044/0 (socket says 172.21.15.27:52044) 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 -- 172.21.15.27:0/3693958411 learned_addr learned my addr 172.21.15.27:0/3693958411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0ea00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef1019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef06000640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10101780 0x7fef1019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 -- 172.21.15.27:0/3693958411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10101780 msgr2=0x7fef1019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10101780 0x7fef1019b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 -- 172.21.15.27:0/3693958411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 msgr2=0x7fef1019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef1019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 -- 172.21.15.27:0/3693958411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101a2700 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0ea00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef1019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef0f400640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef08009fd0 tx=0x7fef0800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.758+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08017070 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.759+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef08003e00 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.759+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08010460 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.759+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101a29f0 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.759+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101a2f10 con 0x7fef10102140 2024-04-26T21:11:24.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.760+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1010c800 con 0x7fef10102140 2024-04-26T21:11:24.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.762+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef08003970 con 0x7fef10102140 2024-04-26T21:11:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.764+0000 7fef07e00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 0x7feef40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.764+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fef080d0780 con 0x7fef10102140 2024-04-26T21:11:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.764+0000 7fef0ea00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 0x7feef40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.765+0000 7fef0ea00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 0x7feef40a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef00009c20 tx=0x7fef00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:24.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.768+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fef0809cf90 con 0x7fef10102140 2024-04-26T21:11:24.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.979+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef1002dcf0 con 0x7fef10102140 2024-04-26T21:11:24.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.982+0000 7fef07e00640 1 -- 172.21.15.27:0/3693958411 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7fef08085e20 con 0x7fef10102140 2024-04-26T21:11:24.983 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:24.984 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:24.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:24 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/644203441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:24.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:24 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1984211412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 msgr2=0x7feef40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 0x7feef40a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef00009c20 tx=0x7fef00005f90 comp rx=0 tx=0).stop 2024-04-26T21:11:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 msgr2=0x7fef101a2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef08009fd0 tx=0x7fef0800b260 comp rx=0 tx=0).stop 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 shutdown_connections 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7feef40a3cc0 0x7feef40a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef10101780 0x7fef1019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef10102140 0x7fef101a2090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 --2- 172.21.15.27:0/3693958411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef10100a20 0x7fef1019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 >> 172.21.15.27:0/3693958411 conn(0x7fef100fc390 msgr2=0x7fef100fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 shutdown_connections 2024-04-26T21:11:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:24.985+0000 7fef0fe00640 1 -- 172.21.15.27:0/3693958411 wait complete. 2024-04-26T21:11:25.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:24 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/644203441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:25.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:24 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1984211412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:25.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:25.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:25.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:24 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/644203441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:25.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:24 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1984211412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:25.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 -- 172.21.15.27:0/823343440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360108580 msgr2=0x7fd36010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 --2- 172.21.15.27:0/823343440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360108580 0x7fd36010a970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd35802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:25.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 -- 172.21.15.27:0/823343440 shutdown_connections 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 --2- 172.21.15.27:0/823343440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360105c50 0x7fd360108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 --2- 172.21.15.27:0/823343440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360108580 0x7fd36010a970 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 --2- 172.21.15.27:0/823343440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360103320 0x7fd360105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 -- 172.21.15.27:0/823343440 >> 172.21.15.27:0/823343440 conn(0x7fd3600fccd0 msgr2=0x7fd3600ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 -- 172.21.15.27:0/823343440 shutdown_connections 2024-04-26T21:11:25.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.224+0000 7fd366000640 1 -- 172.21.15.27:0/823343440 wait complete. 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.225+0000 7fd366000640 1 Processor -- start 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.225+0000 7fd366000640 1 -- start start 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360105c50 0x7fd36019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360108580 0x7fd3601a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010e270 con 0x7fd360103320 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010e0f0 con 0x7fd360105c50 2024-04-26T21:11:25.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd366000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010e3f0 con 0x7fd360108580 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd364c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd364c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52070/0 (socket says 172.21.15.27:52070) 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.226+0000 7fd364c00640 1 -- 172.21.15.27:0/3607541670 learned_addr learned my addr 172.21.15.27:0/3607541670 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd35fe00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360105c50 0x7fd36019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd365600640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360108580 0x7fd3601a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 -- 172.21.15.27:0/3607541670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360108580 msgr2=0x7fd3601a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360108580 0x7fd3601a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 -- 172.21.15.27:0/3607541670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360105c50 msgr2=0x7fd36019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360105c50 0x7fd36019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 -- 172.21.15.27:0/3607541670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601a2790 con 0x7fd360103320 2024-04-26T21:11:25.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd364c00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d950 tx=0x7fd35000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:25.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.227+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350017070 con 0x7fd360103320 2024-04-26T21:11:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.228+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd350009c30 con 0x7fd360103320 2024-04-26T21:11:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.228+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000f620 con 0x7fd360103320 2024-04-26T21:11:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.228+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601a2a80 con 0x7fd360103320 2024-04-26T21:11:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.228+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a3020 con 0x7fd360103320 2024-04-26T21:11:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.229+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36010a070 con 0x7fd360103320 2024-04-26T21:11:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.232+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd350010460 con 0x7fd360103320 2024-04-26T21:11:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.234+0000 7fd35d600640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 0x7fd3400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.234+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd3500d01e0 con 0x7fd360103320 2024-04-26T21:11:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.234+0000 7fd35fe00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 0x7fd3400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.235+0000 7fd35fe00640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 0x7fd3400a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd36019c080 tx=0x7fd34800a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:25.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.238+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd35009ca50 con 0x7fd360103320 2024-04-26T21:11:25.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.447+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd36002dd00 con 0x7fd360103320 2024-04-26T21:11:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.448+0000 7fd35d600640 1 -- 172.21.15.27:0/3607541670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fd350009de0 con 0x7fd360103320 2024-04-26T21:11:25.449 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:11:25.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 msgr2=0x7fd3400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 0x7fd3400a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd36019c080 tx=0x7fd34800a780 comp rx=0 tx=0).stop 2024-04-26T21:11:25.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 msgr2=0x7fd36019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d950 tx=0x7fd35000de10 comp rx=0 tx=0).stop 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 shutdown_connections 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd3400a3cc0 0x7fd3400a6180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd360108580 0x7fd3601a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd360105c50 0x7fd36019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 --2- 172.21.15.27:0/3607541670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd360103320 0x7fd36019ab70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 >> 172.21.15.27:0/3607541670 conn(0x7fd3600fccd0 msgr2=0x7fd360102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 shutdown_connections 2024-04-26T21:11:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.451+0000 7fd366000640 1 -- 172.21.15.27:0/3607541670 wait complete. 2024-04-26T21:11:25.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.693+0000 7fdad7200640 1 -- 172.21.15.27:0/474085483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 msgr2=0x7fdad8077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.693+0000 7fdad7200640 1 --2- 172.21.15.27:0/474085483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad8077710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009900 tx=0x7fdac802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 -- 172.21.15.27:0/474085483 shutdown_connections 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 --2- 172.21.15.27:0/474085483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad8078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 --2- 172.21.15.27:0/474085483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 0x7fdad8079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 --2- 172.21.15.27:0/474085483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad8077710 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 -- 172.21.15.27:0/474085483 >> 172.21.15.27:0/474085483 conn(0x7fdad80fffe0 msgr2=0x7fdad8102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:25.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 -- 172.21.15.27:0/474085483 shutdown_connections 2024-04-26T21:11:25.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.694+0000 7fdad7200640 1 -- 172.21.15.27:0/474085483 wait complete. 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 Processor -- start 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 -- start start 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 0x7fdad81a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8114900 con 0x7fdad80772f0 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8114780 con 0x7fdad8078ed0 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.695+0000 7fdad7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8114a80 con 0x7fdad8078530 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52100/0 (socket says 172.21.15.27:52100) 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 -- 172.21.15.27:0/3857145029 learned_addr learned my addr 172.21.15.27:0/3857145029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdacfe00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad6800640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 0x7fdad81a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 -- 172.21.15.27:0/3857145029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 msgr2=0x7fdad819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 -- 172.21.15.27:0/3857145029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 msgr2=0x7fdad81a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 0x7fdad81a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 -- 172.21.15.27:0/3857145029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad81a6c10 con 0x7fdad80772f0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdacfe00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.696+0000 7fdad5e00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031460 tx=0x7fdac8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.697+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac803f070 con 0x7fdad80772f0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.697+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdac8031da0 con 0x7fdad80772f0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.697+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035030 con 0x7fdad80772f0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.697+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad81a6ea0 con 0x7fdad80772f0 2024-04-26T21:11:25.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.697+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a72d0 con 0x7fdad80772f0 2024-04-26T21:11:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.698+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad81113e0 con 0x7fdad80772f0 2024-04-26T21:11:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.700+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdac8042070 con 0x7fdad80772f0 2024-04-26T21:11:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.702+0000 7fdacea00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 0x7fdabc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.702+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fdac80f0fe0 con 0x7fdad80772f0 2024-04-26T21:11:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.702+0000 7fdacfe00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 0x7fdabc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.703+0000 7fdacfe00640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 0x7fdabc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdad81a0500 tx=0x7fdac0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.706+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdac80bd810 con 0x7fdad80772f0 2024-04-26T21:11:25.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:25 smithi027 ceph-mon[28085]: pgmap v421: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:25.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:25 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3693958411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:25.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:25 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3607541670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:25.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.917+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdad802dd40 con 0x7fdad80772f0 2024-04-26T21:11:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.918+0000 7fdacea00640 1 -- 172.21.15.27:0/3857145029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fdac80bd110 con 0x7fdad80772f0 2024-04-26T21:11:25.919 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 msgr2=0x7fdabc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 0x7fdabc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdad81a0500 tx=0x7fdac0005d30 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 msgr2=0x7fdad819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031460 tx=0x7fdac8003d50 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 shutdown_connections 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdabc0a3cc0 0x7fdabc0a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad8078530 0x7fdad819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad8078ed0 0x7fdad81a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 --2- 172.21.15.27:0/3857145029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad80772f0 0x7fdad819eff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.920+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 >> 172.21.15.27:0/3857145029 conn(0x7fdad80fffe0 msgr2=0x7fdad8103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.921+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 shutdown_connections 2024-04-26T21:11:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:25.921+0000 7fdad7200640 1 -- 172.21.15.27:0/3857145029 wait complete. 2024-04-26T21:11:25.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:11:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:25 smithi204 ceph-mon[32145]: pgmap v421: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:25 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3693958411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:26.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:25 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3607541670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:25 smithi094 ceph-mon[31273]: pgmap v421: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:25 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3693958411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:25 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3607541670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:26.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 -- 172.21.15.27:0/3202611775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50100340 msgr2=0x7fcf50100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 --2- 172.21.15.27:0/3202611775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50100340 0x7fcf50100760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002a410 comp rx=0 tx=0).stop 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 -- 172.21.15.27:0/3202611775 shutdown_connections 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 --2- 172.21.15.27:0/3202611775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 0x7fcf50107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 --2- 172.21.15.27:0/3202611775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50108450 0x7fcf5010a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 --2- 172.21.15.27:0/3202611775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50100340 0x7fcf50100760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.166+0000 7fcf55200640 1 -- 172.21.15.27:0/3202611775 >> 172.21.15.27:0/3202611775 conn(0x7fcf500fbf00 msgr2=0x7fcf500fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.167+0000 7fcf55200640 1 -- 172.21.15.27:0/3202611775 shutdown_connections 2024-04-26T21:11:26.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.167+0000 7fcf55200640 1 -- 172.21.15.27:0/3202611775 wait complete. 2024-04-26T21:11:26.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 Processor -- start 2024-04-26T21:11:26.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 -- start start 2024-04-26T21:11:26.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 0x7fcf50198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 0x7fcf50198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010e180 con 0x7fcf50108450 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010e000 con 0x7fcf50100340 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf55200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010e300 con 0x7fcf50100ca0 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52112/0 (socket says 172.21.15.27:52112) 2024-04-26T21:11:26.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.168+0000 7fcf4fe00640 1 -- 172.21.15.27:0/1887477322 learned_addr learned my addr 172.21.15.27:0/1887477322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf47e00640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 0x7fcf50198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4f400640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 0x7fcf50198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 -- 172.21.15.27:0/1887477322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 msgr2=0x7fcf50198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 0x7fcf50198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 -- 172.21.15.27:0/1887477322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 msgr2=0x7fcf50198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 0x7fcf50198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 -- 172.21.15.27:0/1887477322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf501a0540 con 0x7fcf50108450 2024-04-26T21:11:26.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4f400640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 0x7fcf50198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4fe00640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eaa0 tx=0x7fcf4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800cdb0 con 0x7fcf50108450 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf501a0830 con 0x7fcf50108450 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf48005020 con 0x7fcf50108450 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.169+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480116d0 con 0x7fcf50108450 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.170+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf501a0d30 con 0x7fcf50108450 2024-04-26T21:11:26.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.171+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf50106d20 con 0x7fcf50108450 2024-04-26T21:11:26.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.173+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf480051d0 con 0x7fcf50108450 2024-04-26T21:11:26.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.174+0000 7fcf4cc00640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 0x7fcf340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.174+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf480d0900 con 0x7fcf50108450 2024-04-26T21:11:26.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.174+0000 7fcf4f400640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 0x7fcf340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.175+0000 7fcf4f400640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 0x7fcf340a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf4002a930 tx=0x7fcf40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:26.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.178+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf48016030 con 0x7fcf50108450 2024-04-26T21:11:26.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.386+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf50063530 con 0x7fcf50108450 2024-04-26T21:11:26.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.387+0000 7fcf4cc00640 1 -- 172.21.15.27:0/1887477322 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fcf4809d110 con 0x7fcf50108450 2024-04-26T21:11:26.389 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 msgr2=0x7fcf340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 0x7fcf340a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf4002a930 tx=0x7fcf40005fd0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 msgr2=0x7fcf5019fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eaa0 tx=0x7fcf4800ef60 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 shutdown_connections 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.390+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf340a3c40 0x7fcf340a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf50100ca0 0x7fcf50198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf50108450 0x7fcf5019fed0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 --2- 172.21.15.27:0/1887477322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf50100340 0x7fcf50198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 >> 172.21.15.27:0/1887477322 conn(0x7fcf500fbf00 msgr2=0x7fcf50103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 shutdown_connections 2024-04-26T21:11:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.391+0000 7fcf55200640 1 -- 172.21.15.27:0/1887477322 wait complete. 2024-04-26T21:11:26.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.629+0000 7fca8b000640 1 -- 172.21.15.27:0/990381962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84103940 msgr2=0x7fca84103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.629+0000 7fca8b000640 1 --2- 172.21.15.27:0/990381962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84103940 0x7fca84103dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca70009900 tx=0x7fca7002a410 comp rx=0 tx=0).stop 2024-04-26T21:11:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 -- 172.21.15.27:0/990381962 shutdown_connections 2024-04-26T21:11:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 --2- 172.21.15.27:0/990381962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84104300 0x7fca8410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 --2- 172.21.15.27:0/990381962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84103940 0x7fca84103dc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 --2- 172.21.15.27:0/990381962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84102750 0x7fca84102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 -- 172.21.15.27:0/990381962 >> 172.21.15.27:0/990381962 conn(0x7fca840fdcc0 msgr2=0x7fca84100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 -- 172.21.15.27:0/990381962 shutdown_connections 2024-04-26T21:11:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.630+0000 7fca8b000640 1 -- 172.21.15.27:0/990381962 wait complete. 2024-04-26T21:11:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 Processor -- start 2024-04-26T21:11:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 -- start start 2024-04-26T21:11:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84102750 0x7fca8419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103940 0x7fca8419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca8410e130 con 0x7fca84104300 2024-04-26T21:11:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca8410dfb0 con 0x7fca84102750 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca8410e2b0 con 0x7fca84103940 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.631+0000 7fca8a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52136/0 (socket says 172.21.15.27:52136) 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 -- 172.21.15.27:0/2108718378 learned_addr learned my addr 172.21.15.27:0/2108718378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca89c00640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84102750 0x7fca8419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca89200640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103940 0x7fca8419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 -- 172.21.15.27:0/2108718378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103940 msgr2=0x7fca8419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103940 0x7fca8419b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 -- 172.21.15.27:0/2108718378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84102750 msgr2=0x7fca8419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84102750 0x7fca8419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 -- 172.21.15.27:0/2108718378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca841a26e0 con 0x7fca84104300 2024-04-26T21:11:26.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca8a600640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00eaa0 tx=0x7fca7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.632+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00cdb0 con 0x7fca84104300 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.633+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca7c005020 con 0x7fca84104300 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.633+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca841a29d0 con 0x7fca84104300 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.633+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0116d0 con 0x7fca84104300 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.633+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca841a2ed0 con 0x7fca84104300 2024-04-26T21:11:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.634+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca841a3160 con 0x7fca84104300 2024-04-26T21:11:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.636+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fca7c0051d0 con 0x7fca84104300 2024-04-26T21:11:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.637+0000 7fca82000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 0x7fca6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:26.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.638+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fca7c0d0bd0 con 0x7fca84104300 2024-04-26T21:11:26.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.638+0000 7fca89c00640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 0x7fca6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:26.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.639+0000 7fca89c00640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 0x7fca6c0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca78005fd0 tx=0x7fca78005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:26.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.642+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fca7c016030 con 0x7fca84104300 2024-04-26T21:11:26.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.892+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca841a33d0 con 0x7fca84104300 2024-04-26T21:11:26.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.894+0000 7fca82000640 1 -- 172.21.15.27:0/2108718378 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7fca7c09d3e0 con 0x7fca84104300 2024-04-26T21:11:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:26.896 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:11:26.896 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:11:26.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 msgr2=0x7fca6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 0x7fca6c0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca78005fd0 tx=0x7fca78005850 comp rx=0 tx=0).stop 2024-04-26T21:11:26.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 msgr2=0x7fca841a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:26.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00eaa0 tx=0x7fca7c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 shutdown_connections 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fca6c0a3c40 0x7fca6c0a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103940 0x7fca8419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca84104300 0x7fca841a2070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 --2- 172.21.15.27:0/2108718378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca84102750 0x7fca8419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 >> 172.21.15.27:0/2108718378 conn(0x7fca840fdcc0 msgr2=0x7fca84109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 shutdown_connections 2024-04-26T21:11:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:26.897+0000 7fca8b000640 1 -- 172.21.15.27:0/2108718378 wait complete. 2024-04-26T21:11:26.919 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:26.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:26 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3857145029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:26.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:26 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1887477322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:27.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:26 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3857145029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:27.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:26 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1887477322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:27.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:26 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3857145029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:27.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:26 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1887477322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 -- 172.21.15.27:0/1037293571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 msgr2=0x7f6090107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 --2- 172.21.15.27:0/1037293571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f6090107f10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6080009ca0 tx=0x7f608002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 -- 172.21.15.27:0/1037293571 shutdown_connections 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 --2- 172.21.15.27:0/1037293571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6090108450 0x7f609010a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 --2- 172.21.15.27:0/1037293571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f6090107f10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 --2- 172.21.15.27:0/1037293571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60901031f0 0x7f60901055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 -- 172.21.15.27:0/1037293571 >> 172.21.15.27:0/1037293571 conn(0x7f60900fcbc0 msgr2=0x7f60900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 -- 172.21.15.27:0/1037293571 shutdown_connections 2024-04-26T21:11:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.140+0000 7f608fe00640 1 -- 172.21.15.27:0/1037293571 wait complete. 2024-04-26T21:11:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.141+0000 7f608fe00640 1 Processor -- start 2024-04-26T21:11:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.141+0000 7f608fe00640 1 -- start start 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60901031f0 0x7f6090102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6090108450 0x7f60901016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f609010e1b0 con 0x7f6090105b20 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f609010e030 con 0x7f60901031f0 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f609010e330 con 0x7f6090108450 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52150/0 (socket says 172.21.15.27:52150) 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608e000640 1 -- 172.21.15.27:0/303953409 learned_addr learned my addr 172.21.15.27:0/303953409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608ea00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60901031f0 0x7f6090102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.142+0000 7f608f400640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6090108450 0x7f60901016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 -- 172.21.15.27:0/303953409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6090108450 msgr2=0x7f60901016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6090108450 0x7f60901016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 -- 172.21.15.27:0/303953409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60901031f0 msgr2=0x7f6090102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60901031f0 0x7f6090102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 -- 172.21.15.27:0/303953409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6090101f70 con 0x7f6090105b20 2024-04-26T21:11:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608e000640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6080009430 tx=0x7f6080003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080004040 con 0x7f6090105b20 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6080004670 con 0x7f6090105b20 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080030da0 con 0x7f6090105b20 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.143+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60901a2a10 con 0x7f6090105b20 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.144+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60901a2f60 con 0x7f6090105b20 2024-04-26T21:11:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.145+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60900fe6b0 con 0x7f6090105b20 2024-04-26T21:11:27.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.147+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60800305c0 con 0x7f6090105b20 2024-04-26T21:11:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.148+0000 7f6087400640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 0x7f60740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.148+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f60800f1c30 con 0x7f6090105b20 2024-04-26T21:11:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.149+0000 7f608ea00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 0x7f60740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.149+0000 7f608ea00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 0x7f60740a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6078005fd0 tx=0x7f6078005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.153+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6080004820 con 0x7f6090105b20 2024-04-26T21:11:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.363+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f609002dcf0 con 0x7f6090105b20 2024-04-26T21:11:27.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.368+0000 7f6087400640 1 -- 172.21.15.27:0/303953409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7f60800be550 con 0x7f6090105b20 2024-04-26T21:11:27.369 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:27.370 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 msgr2=0x7f60740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 0x7f60740a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6078005fd0 tx=0x7f6078005850 comp rx=0 tx=0).stop 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 msgr2=0x7f60901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6080009430 tx=0x7f6080003ca0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 shutdown_connections 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f60740a3cc0 0x7f60740a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6090108450 0x7f60901016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090105b20 0x7f60901011a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 --2- 172.21.15.27:0/303953409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60901031f0 0x7f6090102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 >> 172.21.15.27:0/303953409 conn(0x7f60900fcbc0 msgr2=0x7f6090105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:27.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 shutdown_connections 2024-04-26T21:11:27.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.371+0000 7f608fe00640 1 -- 172.21.15.27:0/303953409 wait complete. 2024-04-26T21:11:27.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:27.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 -- 172.21.15.27:0/2547016780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528101670 msgr2=0x7fc528101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 --2- 172.21.15.27:0/2547016780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528101670 0x7fc528101a90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009900 tx=0x7fc51c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 -- 172.21.15.27:0/2547016780 shutdown_connections 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 --2- 172.21.15.27:0/2547016780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528103220 0x7fc528109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 --2- 172.21.15.27:0/2547016780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 0x7fc528102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 --2- 172.21.15.27:0/2547016780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528101670 0x7fc528101a90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 -- 172.21.15.27:0/2547016780 >> 172.21.15.27:0/2547016780 conn(0x7fc5280fcbc0 msgr2=0x7fc5280ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 -- 172.21.15.27:0/2547016780 shutdown_connections 2024-04-26T21:11:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.598+0000 7fc52ea00640 1 -- 172.21.15.27:0/2547016780 wait complete. 2024-04-26T21:11:27.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.599+0000 7fc52ea00640 1 Processor -- start 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.599+0000 7fc52ea00640 1 -- start start 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 0x7fc52819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 0x7fc52819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc52810d0a0 con 0x7fc528103220 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc52810cf20 con 0x7fc528102860 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc52810d220 con 0x7fc528101670 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52164/0 (socket says 172.21.15.27:52164) 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 -- 172.21.15.27:0/235324603 learned_addr learned my addr 172.21.15.27:0/235324603 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52d600640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 0x7fc52819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52cc00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 0x7fc52819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 -- 172.21.15.27:0/235324603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 msgr2=0x7fc52819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 0x7fc52819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 -- 172.21.15.27:0/235324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 msgr2=0x7fc52819b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 0x7fc52819b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52e000640 1 -- 172.21.15.27:0/235324603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281a2780 con 0x7fc528103220 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.600+0000 7fc52d600640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 0x7fc52819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc52e000640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc52000eaa0 tx=0x7fc52000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:27.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000cdb0 con 0x7fc528103220 2024-04-26T21:11:27.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc520005020 con 0x7fc528103220 2024-04-26T21:11:27.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281a2a70 con 0x7fc528103220 2024-04-26T21:11:27.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200116d0 con 0x7fc528103220 2024-04-26T21:11:27.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.601+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a2fc0 con 0x7fc528103220 2024-04-26T21:11:27.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.603+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5281a3250 con 0x7fc528103220 2024-04-26T21:11:27.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.604+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc520004a40 con 0x7fc528103220 2024-04-26T21:11:27.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.607+0000 7fc526000640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 0x7fc5100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:27.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.607+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc5200d0820 con 0x7fc528103220 2024-04-26T21:11:27.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.607+0000 7fc52d600640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 0x7fc5100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:27.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.608+0000 7fc52d600640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 0x7fc5100a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc51c02a910 tx=0x7fc51c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:27.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.612+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc520016030 con 0x7fc528103220 2024-04-26T21:11:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.865+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5281a34c0 con 0x7fc528103220 2024-04-26T21:11:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.866+0000 7fc526000640 1 -- 172.21.15.27:0/235324603 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7fc52009d030 con 0x7fc528103220 2024-04-26T21:11:27.868 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:27.868 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:11:27.868 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:11:27.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 msgr2=0x7fc5100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 0x7fc5100a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc51c02a910 tx=0x7fc51c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 msgr2=0x7fc5281a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc52000eaa0 tx=0x7fc52000ef60 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 shutdown_connections 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fc5100a3c40 0x7fc5100a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc528103220 0x7fc5281a2110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528102860 0x7fc52819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 --2- 172.21.15.27:0/235324603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc528101670 0x7fc52819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 >> 172.21.15.27:0/235324603 conn(0x7fc5280fcbc0 msgr2=0x7fc528069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.869+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 shutdown_connections 2024-04-26T21:11:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:27.870+0000 7fc52ea00640 1 -- 172.21.15.27:0/235324603 wait complete. 2024-04-26T21:11:27.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:09:40.863524+0000', 'modified': '2024-04-26T21:09:49.487318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34472, 'mds_2': 34430, 'mds_3': 34406, 'mds_4': 14640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6839/4118119230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 4118119230}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 4118119230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.27:6841/1078046722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 1078046722}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 1078046722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/3519578339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 3519578339}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 3519578339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1241203213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1241203213}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1241203213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/743097112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 743097112}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 743097112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6839/1122553339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1122553339}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1122553339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.94:6833/2182356287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6832', 'nonce': 2182356287}, {'type': 'v1', 'addr': '172.21.15.94:6833', 'nonce': 2182356287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.94:6835/410689838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 410689838}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 410689838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.94:6837/396354174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 396354174}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 396354174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6839/12233135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 12233135}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 12233135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34472, 34430, 34517, 34406, 14640]} 2024-04-26T21:11:27.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:11:27.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:11:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:27 smithi027 ceph-mon[28085]: pgmap v422: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:27 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2108718378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:27 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/303953409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:27 smithi204 ceph-mon[32145]: pgmap v422: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:28.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:27 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2108718378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:27 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/303953409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:27 smithi094 ceph-mon[31273]: pgmap v422: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:28.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:27 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2108718378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:27 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/303953409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 -- 172.21.15.27:0/1976967450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675010a5f0 msgr2=0x7f675010ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 --2- 172.21.15.27:0/1976967450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675010a5f0 0x7f675010ca90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6748009ca0 tx=0x7f674802a950 comp rx=0 tx=0).stop 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 -- 172.21.15.27:0/1976967450 shutdown_connections 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 --2- 172.21.15.27:0/1976967450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f675010a5f0 0x7f675010ca90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 --2- 172.21.15.27:0/1976967450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6750075330 0x7f6750075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 --2- 172.21.15.27:0/1976967450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f6750074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 -- 172.21.15.27:0/1976967450 >> 172.21.15.27:0/1976967450 conn(0x7f67500fdcb0 msgr2=0x7f6750100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.088+0000 7f6756200640 1 -- 172.21.15.27:0/1976967450 shutdown_connections 2024-04-26T21:11:28.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.089+0000 7f6756200640 1 -- 172.21.15.27:0/1976967450 wait complete. 2024-04-26T21:11:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 Processor -- start 2024-04-26T21:11:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 -- start start 2024-04-26T21:11:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f675019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675010a5f0 0x7f67501a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6750110370 con 0x7f6750075330 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67501101f0 con 0x7f675010a5f0 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6756200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67501104f0 con 0x7f6750076910 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6754e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6754e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52184/0 (socket says 172.21.15.27:52184) 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.090+0000 7f6754e00640 1 -- 172.21.15.27:0/3634108011 learned_addr learned my addr 172.21.15.27:0/3634108011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f674fe00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f675019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6755800640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675010a5f0 0x7f67501a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 -- 172.21.15.27:0/3634108011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 msgr2=0x7f675019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f675019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 -- 172.21.15.27:0/3634108011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675010a5f0 msgr2=0x7f67501a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675010a5f0 0x7f67501a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 -- 172.21.15.27:0/3634108011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67501a2660 con 0x7f6750075330 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f674fe00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f675019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.091+0000 7f6754e00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f674400c890 tx=0x7f674400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.092+0000 7f674d600640 1 -- 172.21.15.27:0/3634108011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744004220 con 0x7f6750075330 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.092+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67501a2950 con 0x7f6750075330 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.092+0000 7f674d600640 1 -- 172.21.15.27:0/3634108011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6744007e40 con 0x7f6750075330 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.092+0000 7f674d600640 1 -- 172.21.15.27:0/3634108011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744004a00 con 0x7f6750075330 2024-04-26T21:11:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.092+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a2ea0 con 0x7f6750075330 2024-04-26T21:11:28.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.096+0000 7f675a781740 1 Processor -- start 2024-04-26T21:11:28.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 -- start start 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c98ef2600 con 0x562c98f8ecb0 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c98e8f100 con 0x562c98f8e2c0 2024-04-26T21:11:28.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.097+0000 7f675a781740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c98eee5c0 con 0x562c98f93e20 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f674d600640 1 -- 172.21.15.27:0/3634108011 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67440079b0 con 0x7f6750075330 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:58076/0 (socket says 172.21.15.27:58076) 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 -- 172.21.15.27:0/1797973976 learned_addr learned my addr 172.21.15.27:0/1797973976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f674fe00640 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6754e00640 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 -- 172.21.15.27:0/1797973976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 msgr2=0x562c98f962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 -- 172.21.15.27:0/1797973976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 msgr2=0x562c98f937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.098+0000 7f6755800640 1 -- 172.21.15.27:0/1797973976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c98c042d0 con 0x562c98f8e2c0 2024-04-26T21:11:28.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.099+0000 7f6755800640 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f674802ae70 tx=0x7f674802f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b260672de4ad2175 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.099+0000 7f674d600640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 0x7f67380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.099+0000 7f6742000640 1 -- 172.21.15.27:0/1797973976 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748034050 con 0x562c98f8e2c0 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.100+0000 7f6742000640 1 -- 172.21.15.27:0/1797973976 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674802f960 con 0x562c98f8e2c0 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.100+0000 7f6742000640 1 -- 172.21.15.27:0/1797973976 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480045d0 con 0x562c98f8e2c0 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.100+0000 7f674d600640 1 -- 172.21.15.27:0/3634108011 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f67440d0320 con 0x7f6750075330 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.100+0000 7f6754e00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 0x7f67380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- 172.21.15.27:0/1797973976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 msgr2=0x562c98f8e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f6754e00640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 0x7f67380a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x562c98f8f720 tx=0x7f67440d3f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f674802ae70 tx=0x7f674802f4e0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- 172.21.15.27:0/1797973976 shutdown_connections 2024-04-26T21:11:28.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 --2- 172.21.15.27:0/1797973976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f8e6e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- 172.21.15.27:0/1797973976 >> 172.21.15.27:0/1797973976 conn(0x562c98f22d10 msgr2=0x562c98f89d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- 172.21.15.27:0/1797973976 shutdown_connections 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- 172.21.15.27:0/1797973976 wait complete. 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.101+0000 7f675a781740 1 -- start start 2024-04-26T21:11:28.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 Processor -- start 2024-04-26T21:11:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c98f96d70 con 0x562c98f8ecb0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c98e79f70 con 0x562c98f8e2c0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c98f96ef0 con 0x562c98f93e20 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f675a781740 10 client.0 _ll_register_callbacks cb 0x562c98ee8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.102+0000 7f674fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52212/0 (socket says 172.21.15.27:52212) 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 -- 172.21.15.27:0/1981170264 learned_addr learned my addr 172.21.15.27:0/1981170264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f6755800640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f6755800640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 -- 172.21.15.27:0/1981170264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 msgr2=0x562c98f9fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f9fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 -- 172.21.15.27:0/1981170264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 msgr2=0x562c98f98660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f98660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 -- 172.21.15.27:0/1981170264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c98cce6b0 con 0x562c98f8ecb0 2024-04-26T21:11:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f6755800640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f9fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f674fe00640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f673c0107d0 tx=0x7f673c010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0453b0fa1519c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.103+0000 7f6736a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c01d060 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f673c00ed40 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674802f960 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f673c018430 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f675a781740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c98fa2c90 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.104+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674802f960 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f673c022450 con 0x562c98f8ecb0 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 10 client.15057 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:11:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 20 client.15057 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c98fa5360 0x562c98fa67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 4 client.15057 mds_command: new command op to 14640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 --> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c98f00210 con 0x562c98fa5360 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.105+0000 7f6755800640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c98fa5360 0x562c98fa67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.106+0000 7f6736a00640 10 client.15057 ms_handle_connect on v2:172.21.15.27:6840/1078046722 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.106+0000 7f6755800640 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c98fa5360 0x562c98fa67d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a653a4d4e1891ab9 server_cookie=fd577a277ecf5beb in_seq=0 out_seq=0 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f6736a00640 1 -- 172.21.15.27:0/1981170264 <== mds.4 v2:172.21.15.27:6840/1078046722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c98f00210 con 0x562c98fa5360 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f6736a00640 10 client.15057 handle_command_reply: tid=0 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 client.15057 shutdown 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 msgr2=0x562c98f98ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f673c0107d0 tx=0x7f673c010c90 comp rx=0 tx=0).stop 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 shutdown_connections 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f9fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c98fa5360 0x562c98fa67d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f98ba0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 --2- 172.21.15.27:0/1981170264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f98660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 >> 172.21.15.27:0/1981170264 conn(0x562c98f22d10 msgr2=0x562c98f88f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f6736a00640 0 client.15057 ms_handle_reset on v2:172.21.15.27:6840/1078046722 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 shutdown_connections 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 1 -- 172.21.15.27:0/1981170264 wait complete. 2024-04-26T21:11:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.107+0000 7f675a781740 20 client.15057 trim_cache size 0 max 16384 2024-04-26T21:11:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 Processor -- start 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 -- start start 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f96210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f90070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa4b10 con 0x562c98f8ecb0 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa51e0 con 0x562c98f8e2c0 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.129+0000 7f675a781740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa58b0 con 0x562c98f93e20 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52216/0 (socket says 172.21.15.27:52216) 2024-04-26T21:11:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 -- 172.21.15.27:0/1709646654 learned_addr learned my addr 172.21.15.27:0/1709646654 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f6755800640 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 -- 172.21.15.27:0/1709646654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 msgr2=0x562c98f90070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f90070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 -- 172.21.15.27:0/1709646654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 msgr2=0x562c98f96210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f96210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 -- 172.21.15.27:0/1709646654 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c98f8f090 con 0x562c98f8ecb0 2024-04-26T21:11:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.130+0000 7f674fe00640 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f673c022890 tx=0x7f673c022de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b6e3f79d895a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.131+0000 7f6741600640 1 -- 172.21.15.27:0/1709646654 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c010270 con 0x562c98f8ecb0 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.131+0000 7f6741600640 1 -- 172.21.15.27:0/1709646654 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f673c00eb40 con 0x562c98f8ecb0 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.131+0000 7f6741600640 1 -- 172.21.15.27:0/1709646654 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0054a0 con 0x562c98f8ecb0 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- 172.21.15.27:0/1709646654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 msgr2=0x562c98f9fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f673c022890 tx=0x7f673c022de0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- 172.21.15.27:0/1709646654 shutdown_connections 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f93e20 0x562c98f90070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f8ecb0 0x562c98f9fba0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 --2- 172.21.15.27:0/1709646654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8e2c0 0x562c98f96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- 172.21.15.27:0/1709646654 >> 172.21.15.27:0/1709646654 conn(0x562c98f22d10 msgr2=0x562c98f84df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- 172.21.15.27:0/1709646654 shutdown_connections 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- 172.21.15.27:0/1709646654 wait complete. 2024-04-26T21:11:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 -- start start 2024-04-26T21:11:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.132+0000 7f675a781740 1 Processor -- start 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f8e2c0 0x562c98f9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8ecb0 0x562c98f9bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa6650 con 0x562c98f93e20 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa5f80 con 0x562c98f8ecb0 2024-04-26T21:11:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c98fa6d20 con 0x562c98f8e2c0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f675a781740 10 client.0 _ll_register_callbacks cb 0x7f673c00b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f6755800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f6755800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52228/0 (socket says 172.21.15.27:52228) 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f6755800640 1 -- 172.21.15.27:0/2795376383 learned_addr learned my addr 172.21.15.27:0/2795376383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f674fe00640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8ecb0 0x562c98f9bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.133+0000 7f6755800640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f8e2c0 0x562c98f9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 -- 172.21.15.27:0/2795376383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f8e2c0 msgr2=0x562c98f9b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f8e2c0 0x562c98f9b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 -- 172.21.15.27:0/2795376383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8ecb0 msgr2=0x562c98f9bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8ecb0 0x562c98f9bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 -- 172.21.15.27:0/2795376383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c98f9f170 con 0x562c98f93e20 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6755800640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x562c98f8f510 tx=0x7f67480093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db375a62b1443af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6736a00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674802fed0 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f673c00eb40 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.134+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f674802f960 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6748060050 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c9900b650 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f673c00eb40 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6748041070 con 0x562c98f93e20 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 10 client.15063 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 20 client.15063 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c9900fe80 0x562c9900c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 4 client.15063 mds_command: new command op to 14640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.135+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 --> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] -- command(tid 0: {"prefix": "status"}) -- 0x562c98efa080 con 0x562c9900fe80 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.136+0000 7f674fe00640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c9900fe80 0x562c9900c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.136+0000 7f674fe00640 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c9900fe80 0x562c9900c2c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3cb9ffbabba15ea5 server_cookie=bad4f0ee78da7d46 in_seq=0 out_seq=0 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.136+0000 7f6736a00640 10 client.15063 ms_handle_connect on v2:172.21.15.27:6840/1078046722 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f6736a00640 1 -- 172.21.15.27:0/2795376383 <== mds.4 v2:172.21.15.27:6840/1078046722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562c98efa080 con 0x562c9900fe80 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f6736a00640 10 client.15063 handle_command_reply: tid=0 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 client.15063 shutdown 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 msgr2=0x562c98f9eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x562c98f8f510 tx=0x7f67480093c0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 shutdown_connections 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x562c9900fe80 0x562c9900c2c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c98f93e20 0x562c98f9eb80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c98f8ecb0 0x562c98f9bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f6736a00640 0 client.15063 ms_handle_reset on v2:172.21.15.27:6840/1078046722 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 --2- 172.21.15.27:0/2795376383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c98f8e2c0 0x562c98f9b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 >> 172.21.15.27:0/2795376383 conn(0x562c98f22d10 msgr2=0x562c98f90070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 shutdown_connections 2024-04-26T21:11:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 1 -- 172.21.15.27:0/2795376383 wait complete. 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.137+0000 7f675a781740 20 client.15063 trim_cache size 0 max 16384 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 14640, 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:11:28.142 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 99.659491935999995, 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 115.97932670100001 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 msgr2=0x7f67380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 0x7f67380a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x562c98f8f720 tx=0x7f67440d3f50 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 msgr2=0x7f675019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f674400c890 tx=0x7f674400cd50 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 shutdown_connections 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f67380a3c40 0x7f67380a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6750076910 0x7f675019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675010a5f0 0x7f67501a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 --2- 172.21.15.27:0/3634108011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6750075330 0x7f675019aa40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 >> 172.21.15.27:0/3634108011 conn(0x7f67500fdcb0 msgr2=0x7f6750106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 shutdown_connections 2024-04-26T21:11:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.138+0000 7f6756200640 1 -- 172.21.15.27:0/3634108011 wait complete. 2024-04-26T21:11:28.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:11:28.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.359+0000 7f6647200640 1 -- 172.21.15.27:0/3222394676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 msgr2=0x7f6640103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.359+0000 7f6647200640 1 --2- 172.21.15.27:0/3222394676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640103dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6630009900 tx=0x7f663002a410 comp rx=0 tx=0).stop 2024-04-26T21:11:28.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 -- 172.21.15.27:0/3222394676 shutdown_connections 2024-04-26T21:11:28.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 --2- 172.21.15.27:0/3222394676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640104310 0x7f664010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 --2- 172.21.15.27:0/3222394676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640103dd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 --2- 172.21.15.27:0/3222394676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640102760 0x7f6640102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 -- 172.21.15.27:0/3222394676 >> 172.21.15.27:0/3222394676 conn(0x7f66400fdcb0 msgr2=0x7f6640100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 -- 172.21.15.27:0/3222394676 shutdown_connections 2024-04-26T21:11:28.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.360+0000 7f6647200640 1 -- 172.21.15.27:0/3222394676 wait complete. 2024-04-26T21:11:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 Processor -- start 2024-04-26T21:11:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 -- start start 2024-04-26T21:11:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640102760 0x7f66401988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640104310 0x7f664019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f664010e140 con 0x7f6640103950 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f664010dfc0 con 0x7f6640102760 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.361+0000 7f6647200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f664010e2c0 con 0x7f6640104310 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52256/0 (socket says 172.21.15.27:52256) 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 -- 172.21.15.27:0/2831817202 learned_addr learned my addr 172.21.15.27:0/2831817202 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645e00640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640102760 0x7f66401988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6646800640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640104310 0x7f664019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 -- 172.21.15.27:0/2831817202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640104310 msgr2=0x7f664019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640104310 0x7f664019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 -- 172.21.15.27:0/2831817202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640102760 msgr2=0x7f66401988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640102760 0x7f66401988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.362+0000 7f6645400640 1 -- 172.21.15.27:0/2831817202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66401a04f0 con 0x7f6640103950 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f6645400640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6630009230 tx=0x7f66300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f663aa00640 1 -- 172.21.15.27:0/2831817202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663003f030 con 0x7f6640103950 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66401a0780 con 0x7f6640103950 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f663aa00640 1 -- 172.21.15.27:0/2831817202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6630003c90 con 0x7f6640103950 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f663aa00640 1 -- 172.21.15.27:0/2831817202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035070 con 0x7f6640103950 2024-04-26T21:11:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.363+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401a0cc0 con 0x7f6640103950 2024-04-26T21:11:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.366+0000 7f664b771740 1 Processor -- start 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f663aa00640 1 -- 172.21.15.27:0/2831817202 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6630042030 con 0x7f6640103950 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 -- start start 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a72c0 0x55da1a6a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55da1a60b600 con 0x55da1a6ace20 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da1a5a8100 con 0x55da1a6a72c0 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f664b771740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6075c0 con 0x55da1a6a7cb0 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52266/0 (socket says 172.21.15.27:52266) 2024-04-26T21:11:28.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.367+0000 7f6645400640 1 -- 172.21.15.27:0/3034682848 learned_addr learned my addr 172.21.15.27:0/3034682848 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6646800640 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a72c0 0x55da1a6a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645e00640 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 -- 172.21.15.27:0/3034682848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 msgr2=0x55da1a6ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 -- 172.21.15.27:0/3034682848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a72c0 msgr2=0x55da1a6a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a72c0 0x55da1a6a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 -- 172.21.15.27:0/3034682848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1a31d2d0 con 0x55da1a6ace20 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f663aa00640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 0x7f66280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.368+0000 7f6645400640 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f66300ee670 tx=0x7f66300f7470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81817cdab2553db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.369+0000 7f663aa00640 1 -- 172.21.15.27:0/2831817202 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f66300f0d90 con 0x7f6640103950 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.369+0000 7f6646800640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 0x7f66280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.369+0000 7f662f000640 1 -- 172.21.15.27:0/3034682848 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300b91a0 con 0x55da1a6ace20 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.369+0000 7f662f000640 1 -- 172.21.15.27:0/3034682848 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6630003e40 con 0x55da1a6ace20 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.369+0000 7f662f000640 1 -- 172.21.15.27:0/3034682848 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300bc860 con 0x55da1a6ace20 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- 172.21.15.27:0/3034682848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 msgr2=0x55da1a6af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f66300ee670 tx=0x7f66300f7470 comp rx=0 tx=0).stop 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f6646800640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 0x7f66280a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55da1a6a8510 tx=0x7f663c009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- 172.21.15.27:0/3034682848 shutdown_connections 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6af2b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 --2- 172.21.15.27:0/3034682848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a72c0 0x55da1a6a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- 172.21.15.27:0/3034682848 >> 172.21.15.27:0/3034682848 conn(0x55da1a63bd10 msgr2=0x55da1a6a2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- 172.21.15.27:0/3034682848 shutdown_connections 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- 172.21.15.27:0/3034682848 wait complete. 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.370+0000 7f664b771740 1 -- start start 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 Processor -- start 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6afb90 con 0x55da1a6a72c0 2024-04-26T21:11:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da1a592f70 con 0x55da1a6ace20 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6afd10 con 0x55da1a6a7cb0 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f664b771740 10 client.0 _ll_register_callbacks cb 0x55da1a601480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f6645e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f6645e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52282/0 (socket says 172.21.15.27:52282) 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.371+0000 7f6645e00640 1 -- 172.21.15.27:0/3412671735 learned_addr learned my addr 172.21.15.27:0/3412671735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6646800640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645400640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 -- 172.21.15.27:0/3412671735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 msgr2=0x55da1a6b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 -- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 msgr2=0x55da1a6bce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6bce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 -- 172.21.15.27:0/3412671735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1a3e76b0 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6645e00640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f663400be10 tx=0x7f66340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f227e576e9f803ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6623400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634004230 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f66340043d0 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.372+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da1a619210 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66340035f0 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da1a6bff30 con 0x55da1a6a72c0 2024-04-26T21:11:28.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55da1a619210 con 0x55da1a6a72c0 2024-04-26T21:11:28.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f663401b770 con 0x55da1a6a72c0 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 10 client.15075 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 20 client.15075 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a6c2670 0x55da1a6c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 4 client.15075 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.373+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 --> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1a613080 con 0x55da1a6c2670 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.374+0000 7f6645400640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a6c2670 0x55da1a6c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.375+0000 7f6645400640 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a6c2670 0x55da1a6c3ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96ca44d417b2c77f server_cookie=73adf65e755d51b8 in_seq=0 out_seq=0 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.375+0000 7f6623400640 10 client.15075 ms_handle_connect on v2:172.21.15.94:6832/2182356287 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f6623400640 1 -- 172.21.15.27:0/3412671735 <== mds.3 v2:172.21.15.94:6832/2182356287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da1a613080 con 0x55da1a6c2670 2024-04-26T21:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f6623400640 10 client.15075 handle_command_reply: tid=0 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 client.15075 shutdown 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 msgr2=0x55da1a6b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f663400be10 tx=0x7f66340117b0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 shutdown_connections 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a6c2670 0x55da1a6c3ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.377+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f6623400640 0 client.15075 ms_handle_reset on v2:172.21.15.94:6832/2182356287 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f664b771740 1 --2- 172.21.15.27:0/3412671735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a72c0 0x55da1a6b5900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 >> 172.21.15.27:0/3412671735 conn(0x55da1a63bd10 msgr2=0x55da1a6a1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 shutdown_connections 2024-04-26T21:11:28.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f664b771740 1 -- 172.21.15.27:0/3412671735 wait complete. 2024-04-26T21:11:28.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.378+0000 7f664b771740 20 client.15075 trim_cache size 0 max 16384 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.398+0000 7f664b771740 1 Processor -- start 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.398+0000 7f664b771740 1 -- start start 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a69ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c1e20 con 0x55da1a6a7cb0 2024-04-26T21:11:28.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c24f0 con 0x55da1a6ace20 2024-04-26T21:11:28.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f664b771740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c2bc0 con 0x55da1a6a72c0 2024-04-26T21:11:28.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52292/0 (socket says 172.21.15.27:52292) 2024-04-26T21:11:28.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 -- 172.21.15.27:0/3968843484 learned_addr learned my addr 172.21.15.27:0/3968843484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6646800640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645e00640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a69ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 -- 172.21.15.27:0/3968843484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 msgr2=0x55da1a69ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a69ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 -- 172.21.15.27:0/3968843484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 msgr2=0x55da1a6a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6645400640 1 -- 172.21.15.27:0/3968843484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1a6ac420 con 0x55da1a6a7cb0 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.399+0000 7f6646800640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.400+0000 7f6645400640 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f66300f4fd0 tx=0x7f6630009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb69d9dbf73d5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.400+0000 7f662e600640 1 -- 172.21.15.27:0/3968843484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300d6070 con 0x55da1a6a7cb0 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.400+0000 7f662e600640 1 -- 172.21.15.27:0/3968843484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f663003b460 con 0x55da1a6a7cb0 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.400+0000 7f662e600640 1 -- 172.21.15.27:0/3968843484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630049040 con 0x55da1a6a7cb0 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- 172.21.15.27:0/3968843484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 msgr2=0x55da1a6bcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f66300f4fd0 tx=0x7f6630009260 comp rx=0 tx=0).stop 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- 172.21.15.27:0/3968843484 shutdown_connections 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6ace20 0x55da1a6a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6a7cb0 0x55da1a6bcda0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 --2- 172.21.15.27:0/3968843484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a69ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- 172.21.15.27:0/3968843484 >> 172.21.15.27:0/3968843484 conn(0x55da1a63bd10 msgr2=0x55da1a6a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- 172.21.15.27:0/3968843484 shutdown_connections 2024-04-26T21:11:28.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- 172.21.15.27:0/3968843484 wait complete. 2024-04-26T21:11:28.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.401+0000 7f664b771740 1 -- start start 2024-04-26T21:11:28.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 Processor -- start 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a6b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c3960 con 0x55da1a6ace20 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c3290 con 0x55da1a6a7cb0 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da1a6c4030 con 0x55da1a6a72c0 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f664b771740 10 client.0 _ll_register_callbacks cb 0x7f663400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f6646800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f6646800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52306/0 (socket says 172.21.15.27:52306) 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.402+0000 7f6646800640 1 -- 172.21.15.27:0/911898059 learned_addr learned my addr 172.21.15.27:0/911898059 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6645400640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6645e00640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a6b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 -- 172.21.15.27:0/911898059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 msgr2=0x55da1a6b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a6b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 -- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a7cb0 msgr2=0x55da1a6b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 -- 172.21.15.27:0/911898059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1a6bc1c0 con 0x55da1a6ace20 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6645e00640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a6b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6646800640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f663c009af0 tx=0x7f663c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d0f52d51de3331 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6623400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c01c970 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f663c01cb10 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.403+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da1a58e860 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f663c01cd10 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da1a728840 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55da1a58e860 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f663c028020 con 0x55da1a6ace20 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 10 client.15081 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 20 client.15081 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a72d530 0x55da1a72c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 4 client.15081 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:11:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.404+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 --> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] -- command(tid 0: {"prefix": "status"}) -- 0x55da1a5dbcb0 con 0x55da1a72d530 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.405+0000 7f6645e00640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a72d530 0x55da1a72c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.406+0000 7f6645e00640 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a72d530 0x55da1a72c240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4c79294c1b512a3 server_cookie=86be7be9ec7ae566 in_seq=0 out_seq=0 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.406+0000 7f6623400640 10 client.15081 ms_handle_connect on v2:172.21.15.94:6832/2182356287 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.406+0000 7f6623400640 1 -- 172.21.15.27:0/911898059 <== mds.3 v2:172.21.15.94:6832/2182356287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55da1a5dbcb0 con 0x55da1a72d530 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.406+0000 7f6623400640 10 client.15081 handle_command_reply: tid=0 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.406+0000 7f664b771740 1 client.15081 shutdown 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 msgr2=0x55da1a6bbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f663c009af0 tx=0x7f663c004a40 comp rx=0 tx=0).stop 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 shutdown_connections 2024-04-26T21:11:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x55da1a72d530 0x55da1a72c240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55da1a6ace20 0x55da1a6bbbd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da1a6a7cb0 0x55da1a6b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 --2- 172.21.15.27:0/911898059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da1a6a72c0 0x55da1a6b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f6623400640 0 client.15081 ms_handle_reset on v2:172.21.15.94:6832/2182356287 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 >> 172.21.15.27:0/911898059 conn(0x55da1a63bd10 msgr2=0x55da1a6c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 shutdown_connections 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 1 -- 172.21.15.27:0/911898059 wait complete. 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.407+0000 7f664b771740 20 client.15081 trim_cache size 0 max 16384 2024-04-26T21:11:28.413 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34406, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 100.82824558599999, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 152.32073405200001 2024-04-26T21:11:28.414 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:11:28.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 msgr2=0x7f66280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 0x7f66280a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55da1a6a8510 tx=0x7f663c009740 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 msgr2=0x7f6640198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6630009230 tx=0x7f66300042f0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 shutdown_connections 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f66280a3cc0 0x7f66280a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640104310 0x7f664019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6640103950 0x7f6640198e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 --2- 172.21.15.27:0/2831817202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6640102760 0x7f66401988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 >> 172.21.15.27:0/2831817202 conn(0x7f66400fdcb0 msgr2=0x7f6640106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 shutdown_connections 2024-04-26T21:11:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.408+0000 7f6647200640 1 -- 172.21.15.27:0/2831817202 wait complete. 2024-04-26T21:11:28.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 -- 172.21.15.27:0/2288517552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 msgr2=0x7f0bc4103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2288517552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc4103ff0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009900 tx=0x7f0bb002a410 comp rx=0 tx=0).stop 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 -- 172.21.15.27:0/2288517552 shutdown_connections 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2288517552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4104530 0x7f0bc410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2288517552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc4103ff0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2288517552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4102980 0x7f0bc4102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 -- 172.21.15.27:0/2288517552 >> 172.21.15.27:0/2288517552 conn(0x7f0bc40fdf10 msgr2=0x7f0bc4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 -- 172.21.15.27:0/2288517552 shutdown_connections 2024-04-26T21:11:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.617+0000 7f0bcb800640 1 -- 172.21.15.27:0/2288517552 wait complete. 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 Processor -- start 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 -- start start 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4102980 0x7f0bc419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4104530 0x7f0bc41a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410e360 con 0x7f0bc4103b70 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410e1e0 con 0x7f0bc4104530 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.619+0000 7f0bcb800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410e4e0 con 0x7f0bc4102980 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52324/0 (socket says 172.21.15.27:52324) 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 -- 172.21.15.27:0/2064016854 learned_addr learned my addr 172.21.15.27:0/2064016854 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4102980 0x7f0bc419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc9800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4104530 0x7f0bc41a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 -- 172.21.15.27:0/2064016854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4102980 msgr2=0x7f0bc419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4102980 0x7f0bc419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 -- 172.21.15.27:0/2064016854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4104530 msgr2=0x7f0bc41a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4104530 0x7f0bc41a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 -- 172.21.15.27:0/2064016854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41a28f0 con 0x7f0bc4103b70 2024-04-26T21:11:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.620+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009230 tx=0x7f0bb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.621+0000 7f0bc1600640 1 -- 172.21.15.27:0/2064016854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb003f030 con 0x7f0bc4103b70 2024-04-26T21:11:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.621+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc41a2b80 con 0x7f0bc4103b70 2024-04-26T21:11:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.621+0000 7f0bc1600640 1 -- 172.21.15.27:0/2064016854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb0003c90 con 0x7f0bc4103b70 2024-04-26T21:11:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.621+0000 7f0bc1600640 1 -- 172.21.15.27:0/2064016854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0035070 con 0x7f0bc4103b70 2024-04-26T21:11:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.621+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a30c0 con 0x7f0bc4103b70 2024-04-26T21:11:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.624+0000 7f0bce767740 1 Processor -- start 2024-04-26T21:11:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.624+0000 7f0bce767740 1 -- start start 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.624+0000 7f0bce767740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df40f0 0x55acb9df4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.624+0000 7f0bce767740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bce767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bce767740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acb9d58600 con 0x55acb9df4ae0 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bce767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55acb9cf5100 con 0x55acb9df9c50 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bce767740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55acb9d545c0 con 0x55acb9df40f0 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52330/0 (socket says 172.21.15.27:52330) 2024-04-26T21:11:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 -- 172.21.15.27:0/2554689946 learned_addr learned my addr 172.21.15.27:0/2554689946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc1600640 1 -- 172.21.15.27:0/2064016854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bb0042030 con 0x7f0bc4103b70 2024-04-26T21:11:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc9800640 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df40f0 0x55acb9df4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 -- 172.21.15.27:0/2554689946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df40f0 msgr2=0x55acb9df4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df40f0 0x55acb9df4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 -- 172.21.15.27:0/2554689946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 msgr2=0x55acb9dfc0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfc0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.625+0000 7f0bc8e00640 1 -- 172.21.15.27:0/2554689946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb9a6a2d0 con 0x55acb9df4ae0 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.626+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800bc70 tx=0x7f0bb800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafcfe74e35b2fff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.626+0000 7f0bb6000640 1 -- 172.21.15.27:0/2554689946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8015070 con 0x55acb9df4ae0 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.626+0000 7f0bb6000640 1 -- 172.21.15.27:0/2554689946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb80035f0 con 0x55acb9df4ae0 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.626+0000 7f0bb6000640 1 -- 172.21.15.27:0/2554689946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80116a0 con 0x55acb9df4ae0 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 -- 172.21.15.27:0/2554689946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 msgr2=0x55acb9df95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800bc70 tx=0x7f0bb800f730 comp rx=0 tx=0).stop 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bc1600640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 0x7f0bac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 -- 172.21.15.27:0/2554689946 shutdown_connections 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bc1600640 1 -- 172.21.15.27:0/2064016854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0bb00f0dc0 con 0x7f0bc4103b70 2024-04-26T21:11:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df4ae0 0x55acb9df95e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 --2- 172.21.15.27:0/2554689946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df40f0 0x55acb9df4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 -- 172.21.15.27:0/2554689946 >> 172.21.15.27:0/2554689946 conn(0x55acb9d88d10 msgr2=0x55acb9defb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bc9800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 0x7f0bac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 -- 172.21.15.27:0/2554689946 shutdown_connections 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.627+0000 7f0bce767740 1 -- 172.21.15.27:0/2554689946 wait complete. 2024-04-26T21:11:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 -- start start 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bc9800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 0x7f0bac0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55acb9df5340 tx=0x7f0bbc009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 Processor -- start 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9dfe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9dfeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acb9dfcc00 con 0x55acb9df9c50 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55acb9cdff70 con 0x55acb9df40f0 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.628+0000 7f0bce767740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55acb9dfcd80 con 0x55acb9df4ae0 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bce767740 10 client.0 _ll_register_callbacks cb 0x55acb9d4e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52332/0 (socket says 172.21.15.27:52332) 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 -- 172.21.15.27:0/428828467 learned_addr learned my addr 172.21.15.27:0/428828467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9dfeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9dfe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 -- 172.21.15.27:0/428828467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 msgr2=0x55acb9dfeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9dfeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 -- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 msgr2=0x55acb9dfe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9dfe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.629+0000 7f0bc9800640 1 -- 172.21.15.27:0/428828467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb9b346b0 con 0x55acb9df9c50 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0bc9800640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc012870 tx=0x7f0bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e05c6fd628000b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0baaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00ead0 con 0x55acb9df9c50 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb80035f0 con 0x55acb9df9c50 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0bbc00ec70 con 0x55acb9df9c50 2024-04-26T21:11:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc01f420 con 0x55acb9df9c50 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0bce767740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acb9e08c80 con 0x55acb9df9c50 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.630+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb80035f0 con 0x55acb9df9c50 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bbc00f040 con 0x55acb9df9c50 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 10 client.15093 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 20 client.15093 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e0b2c0 0x55acb9e0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 4 client.15093 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.631+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 --> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb9d66210 con 0x55acb9e0b2c0 2024-04-26T21:11:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.632+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e0b2c0 0x55acb9e0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.632+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e0b2c0 0x55acb9e0c730 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=368e75b82b6c2c18 server_cookie=b844843d537c7c7b in_seq=0 out_seq=0 2024-04-26T21:11:28.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.632+0000 7f0baaa00640 10 client.15093 ms_handle_connect on v2:172.21.15.94:6834/410689838 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0baaa00640 1 -- 172.21.15.27:0/428828467 <== mds.2 v2:172.21.15.94:6834/410689838 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acb9d66210 con 0x55acb9e0b2c0 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0baaa00640 10 client.15093 handle_command_reply: tid=0 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 client.15093 shutdown 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 msgr2=0x55acb9e05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc012870 tx=0x7f0bbc004a40 comp rx=0 tx=0).stop 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 shutdown_connections 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e0b2c0 0x55acb9e0c730 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9dfeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e05b80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 --2- 172.21.15.27:0/428828467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9dfe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0baaa00640 0 client.15093 ms_handle_reset on v2:172.21.15.94:6834/410689838 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 >> 172.21.15.27:0/428828467 conn(0x55acb9d88d10 msgr2=0x55acb9deeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 shutdown_connections 2024-04-26T21:11:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 1 -- 172.21.15.27:0/428828467 wait complete. 2024-04-26T21:11:28.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.635+0000 7f0bce767740 20 client.15093 trim_cache size 0 max 16384 2024-04-26T21:11:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 Processor -- start 2024-04-26T21:11:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 -- start start 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0aa70 con 0x55acb9df40f0 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0b140 con 0x55acb9df9c50 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.655+0000 7f0bce767740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0b810 con 0x55acb9df4ae0 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52334/0 (socket says 172.21.15.27:52334) 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 -- 172.21.15.27:0/1518531239 learned_addr learned my addr 172.21.15.27:0/1518531239 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc9800640 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 -- 172.21.15.27:0/1518531239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 msgr2=0x55acb9e05aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e05aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 -- 172.21.15.27:0/1518531239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 msgr2=0x55acb9dfe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.656+0000 7f0bc8e00640 1 -- 172.21.15.27:0/1518531239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb9df6620 con 0x55acb9df40f0 2024-04-26T21:11:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800fc30 tx=0x7f0bb800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb04a894f19fc718 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bb5600640 1 -- 172.21.15.27:0/1518531239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8015040 con 0x55acb9df40f0 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bb5600640 1 -- 172.21.15.27:0/1518531239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb80054d0 con 0x55acb9df40f0 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bb5600640 1 -- 172.21.15.27:0/1518531239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80265d0 con 0x55acb9df40f0 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bce767740 1 -- 172.21.15.27:0/1518531239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 msgr2=0x55acb9df5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.657+0000 7f0bce767740 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800fc30 tx=0x7f0bb800b850 comp rx=0 tx=0).stop 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 -- 172.21.15.27:0/1518531239 shutdown_connections 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e05aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df9c50 0x55acb9dfe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 --2- 172.21.15.27:0/1518531239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df40f0 0x55acb9df5f40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 -- 172.21.15.27:0/1518531239 >> 172.21.15.27:0/1518531239 conn(0x55acb9d88d10 msgr2=0x55acb9dfe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 -- 172.21.15.27:0/1518531239 shutdown_connections 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 -- 172.21.15.27:0/1518531239 wait complete. 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 -- start start 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.658+0000 7f0bce767740 1 Processor -- start 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9e016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e01c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0c5b0 con 0x55acb9df9c50 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0bee0 con 0x55acb9df40f0 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55acb9e0cc80 con 0x55acb9df4ae0 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bce767740 10 client.0 _ll_register_callbacks cb 0x7f0bbc009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52348/0 (socket says 172.21.15.27:52348) 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 -- 172.21.15.27:0/1184905853 learned_addr learned my addr 172.21.15.27:0/1184905853 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e01c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9e016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 -- 172.21.15.27:0/1184905853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 msgr2=0x55acb9e01c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e01c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 -- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 msgr2=0x55acb9e016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9e016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc9800640 1 -- 172.21.15.27:0/1184905853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb9e05220 con 0x55acb9df9c50 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9e016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.659+0000 7f0bc3e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e01c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0bc9800640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005cd0 tx=0x7f0bbc005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87dbfde8d8b0bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0baaa00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00e930 con 0x55acb9df9c50 2024-04-26T21:11:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb80054d0 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0bbc00ead0 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc00ecd0 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0bce767740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.660+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55acb9e71750 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb80054d0 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bbc045050 con 0x55acb9df9c50 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 10 client.15099 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 20 client.15099 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e763f0 0x55acb9e72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 4 client.15099 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:11:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 --> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] -- command(tid 0: {"prefix": "status"}) -- 0x55acb9d60080 con 0x55acb9e763f0 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.661+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e763f0 0x55acb9e72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.662+0000 7f0bc8e00640 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e763f0 0x55acb9e72830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99cf1a267041abf1 server_cookie=a60b74695be418dc in_seq=0 out_seq=0 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.662+0000 7f0baaa00640 10 client.15099 ms_handle_connect on v2:172.21.15.94:6834/410689838 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0baaa00640 1 -- 172.21.15.27:0/1184905853 <== mds.2 v2:172.21.15.94:6834/410689838 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55acb9d60080 con 0x55acb9e763f0 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0baaa00640 10 client.15099 handle_command_reply: tid=0 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 client.15099 shutdown 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 msgr2=0x55acb9e04c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005cd0 tx=0x7f0bbc005d00 comp rx=0 tx=0).stop 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 shutdown_connections 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55acb9e763f0 0x55acb9e72830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55acb9df4ae0 0x55acb9e01c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55acb9df9c50 0x55acb9e04c30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 --2- 172.21.15.27:0/1184905853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acb9df40f0 0x55acb9e016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 >> 172.21.15.27:0/1184905853 conn(0x55acb9d88d10 msgr2=0x55acb9df2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0baaa00640 0 client.15099 ms_handle_reset on v2:172.21.15.94:6834/410689838 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 shutdown_connections 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.663+0000 7f0bce767740 1 -- 172.21.15.27:0/1184905853 wait complete. 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bce767740 20 client.15099 trim_cache size 0 max 16384 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34430, 2024-04-26T21:11:28.668 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 102.101195113, 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 140.141835375 2024-04-26T21:11:28.669 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 msgr2=0x7f0bac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 0x7f0bac0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55acb9df5340 tx=0x7f0bbc009580 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 msgr2=0x7f0bc419b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009230 tx=0x7f0bb00042f0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 shutdown_connections 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f0bac0a3c40 0x7f0bac0a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc4104530 0x7f0bc41a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bc4103b70 0x7f0bc419b1f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 --2- 172.21.15.27:0/2064016854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4102980 0x7f0bc419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 >> 172.21.15.27:0/2064016854 conn(0x7f0bc40fdf10 msgr2=0x7f0bc4109390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 shutdown_connections 2024-04-26T21:11:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.664+0000 7f0bcb800640 1 -- 172.21.15.27:0/2064016854 wait complete. 2024-04-26T21:11:28.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:11:28.741 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:28 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/235324603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 -- 172.21.15.27:0/1753705700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab01032a0 msgr2=0x7f8ab0105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 --2- 172.21.15.27:0/1753705700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab01032a0 0x7f8ab0105690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009900 tx=0x7f8aa002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 -- 172.21.15.27:0/1753705700 shutdown_connections 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 --2- 172.21.15.27:0/1753705700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab0108500 0x7f8ab010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 --2- 172.21.15.27:0/1753705700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0105bd0 0x7f8ab0107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 --2- 172.21.15.27:0/1753705700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab01032a0 0x7f8ab0105690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 -- 172.21.15.27:0/1753705700 >> 172.21.15.27:0/1753705700 conn(0x7f8ab00fcbb0 msgr2=0x7f8ab00ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.871+0000 7f8aafe00640 1 -- 172.21.15.27:0/1753705700 shutdown_connections 2024-04-26T21:11:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.872+0000 7f8aafe00640 1 -- 172.21.15.27:0/1753705700 wait complete. 2024-04-26T21:11:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.872+0000 7f8aafe00640 1 Processor -- start 2024-04-26T21:11:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 -- start start 2024-04-26T21:11:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab01032a0 0x7f8ab019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 0x7f8ab01a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010e210 con 0x7f8ab0105bd0 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010e090 con 0x7f8ab0108500 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aafe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010e390 con 0x7f8ab01032a0 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52376/0 (socket says 172.21.15.27:52376) 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aa7e00640 1 -- 172.21.15.27:0/4091582925 learned_addr learned my addr 172.21.15.27:0/4091582925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aaea00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab01032a0 0x7f8ab019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aaf400640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 0x7f8ab01a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.873+0000 7f8aa7e00640 1 -- 172.21.15.27:0/4091582925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab01032a0 msgr2=0x7f8ab019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab01032a0 0x7f8ab019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7e00640 1 -- 172.21.15.27:0/4091582925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 msgr2=0x7f8ab01a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 0x7f8ab01a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7e00640 1 -- 172.21.15.27:0/4091582925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a2740 con 0x7f8ab0105bd0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aaf400640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 0x7f8ab01a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c970 tx=0x7f8a9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7400640 1 -- 172.21.15.27:0/4091582925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98007bb0 con 0x7f8ab0105bd0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7400640 1 -- 172.21.15.27:0/4091582925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a98007d50 con 0x7f8ab0105bd0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aa7400640 1 -- 172.21.15.27:0/4091582925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98004e10 con 0x7f8ab0105bd0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a2a30 con 0x7f8ab0105bd0 2024-04-26T21:11:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.874+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a3050 con 0x7f8ab0105bd0 2024-04-26T21:11:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.877+0000 7f8ab8b65740 1 Processor -- start 2024-04-26T21:11:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.877+0000 7f8ab8b65740 1 -- start start 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ab82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c37a14600 con 0x558c37ab5e20 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c379b1100 con 0x558c37ab02c0 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c37a105c0 con 0x558c37ab0cb0 2024-04-26T21:11:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8aaea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.878+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:39894/0 (socket says 172.21.15.27:39894) 2024-04-26T21:11:28.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 -- 172.21.15.27:0/1833839205 learned_addr learned my addr 172.21.15.27:0/1833839205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.27:58270/0 (socket says 172.21.15.27:58270) 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aa7400640 1 -- 172.21.15.27:0/4091582925 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a98005000 con 0x7f8ab0105bd0 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 -- 172.21.15.27:0/1833839205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 msgr2=0x558c37ab06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 -- 172.21.15.27:0/1833839205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 msgr2=0x558c37ab82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ab82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaea00640 1 -- 172.21.15.27:0/1833839205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c377262d0 con 0x558c37ab0cb0 2024-04-26T21:11:28.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.879+0000 7f8aaf400640 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.880+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0031460 tx=0x7f8aa0004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825c3d2561f5ecf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.880+0000 7f8a9fe00640 1 -- 172.21.15.27:0/1833839205 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0035030 con 0x558c37ab0cb0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8a9fe00640 1 -- 172.21.15.27:0/1833839205 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa0003c80 con 0x558c37ab0cb0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8a9fe00640 1 -- 172.21.15.27:0/1833839205 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0031be0 con 0x558c37ab0cb0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8aa7400640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 0x7f8a940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8aa7400640 1 -- 172.21.15.27:0/4091582925 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f8a980d0680 con 0x7f8ab0105bd0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8aaf400640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 0x7f8a940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1833839205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 msgr2=0x558c37ab57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.881+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0031460 tx=0x7f8aa0004390 comp rx=0 tx=0).stop 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1833839205 shutdown_connections 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37ab57b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ab82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1833839205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37ab06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1833839205 >> 172.21.15.27:0/1833839205 conn(0x558c37a44d10 msgr2=0x558c37aabd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8aaf400640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 0x7f8a940a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558c37ab1510 tx=0x7f8aa8007740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1833839205 shutdown_connections 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1833839205 wait complete. 2024-04-26T21:11:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 -- start start 2024-04-26T21:11:28.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.882+0000 7f8ab8b65740 1 Processor -- start 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37aba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab5e20 0x558c37ac1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c37ab8d70 con 0x558c37ab0cb0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c3799bf70 con 0x558c37ab5e20 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c37ab8ef0 con 0x558c37ab02c0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8ab8b65740 10 client.0 _ll_register_callbacks cb 0x558c37a0a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52388/0 (socket says 172.21.15.27:52388) 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 -- 172.21.15.27:0/1271961583 learned_addr learned my addr 172.21.15.27:0/1271961583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37aba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aaf400640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab5e20 0x558c37ac1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 -- 172.21.15.27:0/1271961583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 msgr2=0x558c37aba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37aba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 -- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab5e20 msgr2=0x558c37ac1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab5e20 0x558c37ac1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.883+0000 7f8aa7e00640 1 -- 172.21.15.27:0/1271961583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c377f06b0 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c940 tx=0x7f8a980055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aace1c70d836bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9801d040 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f8a980870c0 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa0003c80 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a9809c7f0 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8ab8b65740 10 client.15108 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c37ac4d70 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.884+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa0003c80 con 0x558c37ab0cb0 2024-04-26T21:11:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8a980a4fd0 con 0x558c37ab0cb0 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 20 client.15108 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37ac73d0 0x558c37ac8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 4 client.15108 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.885+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 --> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c37a22210 con 0x558c37ac73d0 2024-04-26T21:11:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.886+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37ac73d0 0x558c37ac8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.887+0000 7f8aaea00640 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37ac73d0 0x558c37ac8840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43bacb19a9ee0df1 server_cookie=e3517e25691f2100 in_seq=0 out_seq=0 2024-04-26T21:11:28.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.887+0000 7f8a9cc00640 10 client.15108 ms_handle_connect on v2:172.21.15.94:6836/396354174 2024-04-26T21:11:28.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8a9cc00640 1 -- 172.21.15.27:0/1271961583 <== mds.1 v2:172.21.15.94:6836/396354174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c37a22210 con 0x558c37ac73d0 2024-04-26T21:11:28.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8a9cc00640 10 client.15108 handle_command_reply: tid=0 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 client.15108 shutdown 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 msgr2=0x558c37abace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c940 tx=0x7f8a980055b0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 shutdown_connections 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37ac73d0 0x558c37ac8840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab5e20 0x558c37ac1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab0cb0 0x558c37abace0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8a9cc00640 0 client.15108 ms_handle_reset on v2:172.21.15.94:6836/396354174 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/1271961583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37aba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 >> 172.21.15.27:0/1271961583 conn(0x558c37a44d10 msgr2=0x558c37aaaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 shutdown_connections 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 1 -- 172.21.15.27:0/1271961583 wait complete. 2024-04-26T21:11:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.889+0000 7f8ab8b65740 20 client.15108 trim_cache size 0 max 16384 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.912+0000 7f8ab8b65740 1 Processor -- start 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 -- start start 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37ad0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab0cb0 0x558c37ad1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac6b80 con 0x558c37ab5e20 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac7250 con 0x558c37ab0cb0 2024-04-26T21:11:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac7920 con 0x558c37ab02c0 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52404/0 (socket says 172.21.15.27:52404) 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.913+0000 7f8aaf400640 1 -- 172.21.15.27:0/2071017966 learned_addr learned my addr 172.21.15.27:0/2071017966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab0cb0 0x558c37ad1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaea00640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37ad0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 -- 172.21.15.27:0/2071017966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 msgr2=0x558c37ad0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37ad0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 -- 172.21.15.27:0/2071017966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab0cb0 msgr2=0x558c37ad1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab0cb0 0x558c37ad1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 -- 172.21.15.27:0/2071017966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c37abb920 con 0x558c37ab5e20 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaea00640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37ad0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d710 tx=0x7f8aa8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabfb0af52d6521a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.914+0000 7f8a9f400640 1 -- 172.21.15.27:0/2071017966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8010b30 con 0x558c37ab5e20 2024-04-26T21:11:28.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8a9f400640 1 -- 172.21.15.27:0/2071017966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa8021070 con 0x558c37ab5e20 2024-04-26T21:11:28.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8a9f400640 1 -- 172.21.15.27:0/2071017966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa801d420 con 0x558c37ab5e20 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2071017966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 msgr2=0x558c37ac1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d710 tx=0x7f8aa8012000 comp rx=0 tx=0).stop 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2071017966 shutdown_connections 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1ce0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab0cb0 0x558c37ad1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2071017966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab02c0 0x558c37ad0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2071017966 >> 172.21.15.27:0/2071017966 conn(0x558c37a44d10 msgr2=0x558c37ab21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2071017966 shutdown_connections 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.915+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2071017966 wait complete. 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 -- start start 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 Processor -- start 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37abdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37abe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac86c0 con 0x558c37ab5e20 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac7ff0 con 0x558c37ab02c0 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c37ac8d90 con 0x558c37ab0cb0 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.916+0000 7f8ab8b65740 10 client.0 _ll_register_callbacks cb 0x7f8a98007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52416/0 (socket says 172.21.15.27:52416) 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 -- 172.21.15.27:0/2489951081 learned_addr learned my addr 172.21.15.27:0/2489951081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aa7e00640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37abe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaea00640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37abdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 -- 172.21.15.27:0/2489951081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 msgr2=0x558c37abe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37abe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 -- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 msgr2=0x558c37abdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37abdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.917+0000 7f8aaf400640 1 -- 172.21.15.27:0/2489951081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c37ac1a10 con 0x558c37ab5e20 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8aaf400640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d6e0 tx=0x7f8aa8010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905e6252879b02fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8036060 con 0x558c37ab5e20 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa8021070 con 0x558c37ab5e20 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f8aa801d420 con 0x558c37ab5e20 2024-04-26T21:11:28.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8aa8007bb0 con 0x558c37ab5e20 2024-04-26T21:11:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8ab8b65740 10 client.15114 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:28.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.918+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c37b2dd40 con 0x558c37ab5e20 2024-04-26T21:11:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa8021070 con 0x558c37ab5e20 2024-04-26T21:11:28.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8aa803c480 con 0x558c37ab5e20 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 20 client.15114 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37b32580 0x558c37b31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 4 client.15114 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.919+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 --> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] -- command(tid 0: {"prefix": "status"}) -- 0x558c37a1c080 con 0x558c37b32580 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.920+0000 7f8aaea00640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37b32580 0x558c37b31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.920+0000 7f8aaea00640 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37b32580 0x558c37b31290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9da26d23aecf65a5 server_cookie=99d0892f2c7d5f8d in_seq=0 out_seq=0 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.920+0000 7f8a9cc00640 10 client.15114 ms_handle_connect on v2:172.21.15.94:6836/396354174 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8a9cc00640 1 -- 172.21.15.27:0/2489951081 <== mds.1 v2:172.21.15.94:6836/396354174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c37a1c080 con 0x558c37b32580 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8a9cc00640 10 client.15114 handle_command_reply: tid=0 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 client.15114 shutdown 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 msgr2=0x558c37ac1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d6e0 tx=0x7f8aa8010b60 comp rx=0 tx=0).stop 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 shutdown_connections 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x558c37b32580 0x558c37b31290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c37ab0cb0 0x558c37abe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558c37ab5e20 0x558c37ac1420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 --2- 172.21.15.27:0/2489951081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c37ab02c0 0x558c37abdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 >> 172.21.15.27:0/2489951081 conn(0x558c37a44d10 msgr2=0x558c37ab9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8a9cc00640 0 client.15114 ms_handle_reset on v2:172.21.15.94:6836/396354174 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 shutdown_connections 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.921+0000 7f8ab8b65740 1 -- 172.21.15.27:0/2489951081 wait complete. 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8ab8b65740 20 client.15114 trim_cache size 0 max 16384 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:11:28.925 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34472, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 103.37305731799999, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 128.86194657600001 2024-04-26T21:11:28.926 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 msgr2=0x7f8a940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 0x7f8a940a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558c37ab1510 tx=0x7f8aa8007740 comp rx=0 tx=0).stop 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 msgr2=0x7f8ab019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c970 tx=0x7f8a9800ce30 comp rx=0 tx=0).stop 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 shutdown_connections 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f8a940a3cc0 0x7f8a940a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ab0108500 0x7f8ab01a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ab0105bd0 0x7f8ab019b060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 --2- 172.21.15.27:0/4091582925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ab01032a0 0x7f8ab019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 >> 172.21.15.27:0/4091582925 conn(0x7f8ab00fcbb0 msgr2=0x7f8ab0102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 shutdown_connections 2024-04-26T21:11:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:28.922+0000 7f8aafe00640 1 -- 172.21.15.27:0/4091582925 wait complete. 2024-04-26T21:11:28.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:11:29.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:28 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/235324603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:29.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:28 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/235324603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.126+0000 7fbf8de00640 1 -- 172.21.15.27:0/2022553477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88108d90 msgr2=0x7fbf881091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.126+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2022553477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88108d90 0x7fbf881091b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf7802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.126+0000 7fbf8de00640 1 -- 172.21.15.27:0/2022553477 shutdown_connections 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.126+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2022553477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 0x7fbf881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.127+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2022553477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88109f80 0x7fbf8810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.127+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2022553477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88108d90 0x7fbf881091b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.127+0000 7fbf8de00640 1 -- 172.21.15.27:0/2022553477 >> 172.21.15.27:0/2022553477 conn(0x7fbf88077d30 msgr2=0x7fbf8807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.127+0000 7fbf8de00640 1 -- 172.21.15.27:0/2022553477 shutdown_connections 2024-04-26T21:11:29.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.127+0000 7fbf8de00640 1 -- 172.21.15.27:0/2022553477 wait complete. 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 Processor -- start 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 -- start start 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88108d90 0x7fbf8819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 0x7fbf881a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881147c0 con 0x7fbf88109f80 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88114640 con 0x7fbf88108d90 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf8de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88114940 con 0x7fbf8810a940 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54210/0 (socket says 172.21.15.27:54210) 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.128+0000 7fbf7fe00640 1 -- 172.21.15.27:0/2290925294 learned_addr learned my addr 172.21.15.27:0/2290925294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88108d90 0x7fbf8819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf8d400640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 0x7fbf881a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 -- 172.21.15.27:0/2290925294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 msgr2=0x7fbf881a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 0x7fbf881a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 -- 172.21.15.27:0/2290925294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88108d90 msgr2=0x7fbf8819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88108d90 0x7fbf8819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 -- 172.21.15.27:0/2290925294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf881a6ae0 con 0x7fbf88109f80 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000d950 tx=0x7fbf7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.129+0000 7fbf86000640 1 -- 172.21.15.27:0/2290925294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70017070 con 0x7fbf88109f80 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.130+0000 7fbf86000640 1 -- 172.21.15.27:0/2290925294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf70009c30 con 0x7fbf88109f80 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.130+0000 7fbf86000640 1 -- 172.21.15.27:0/2290925294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000f620 con 0x7fbf88109f80 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.130+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf881a6dd0 con 0x7fbf88109f80 2024-04-26T21:11:29.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.130+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a7240 con 0x7fbf88109f80 2024-04-26T21:11:29.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 Processor -- start 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 -- start start 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf86000640 1 -- 172.21.15.27:0/2290925294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf70010460 con 0x7fbf88109f80 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f8e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559862eed600 con 0x559862f8ee20 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559862e8a100 con 0x559862f89cb0 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.133+0000 7fbf92252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559862ee95c0 con 0x559862f892c0 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54216/0 (socket says 172.21.15.27:54216) 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 -- 172.21.15.27:0/801115539 learned_addr learned my addr 172.21.15.27:0/801115539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf8d400640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f8e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 -- 172.21.15.27:0/801115539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 msgr2=0x559862f896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 -- 172.21.15.27:0/801115539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 msgr2=0x559862f8e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f8e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 -- 172.21.15.27:0/801115539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559862bff2d0 con 0x559862f8ee20 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f8e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.134+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbf700cdb80 tx=0x7fbf700d8990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80abf602309a721b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf7dc00640 1 -- 172.21.15.27:0/801115539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700e4070 con 0x559862f8ee20 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf7dc00640 1 -- 172.21.15.27:0/801115539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf700d24e0 con 0x559862f8ee20 2024-04-26T21:11:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf86000640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 0x7fbf6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf86000640 1 -- 172.21.15.27:0/2290925294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbf700d02a0 con 0x7fbf88109f80 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf8d400640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 0x7fbf6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.135+0000 7fbf7dc00640 1 -- 172.21.15.27:0/801115539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700986b0 con 0x559862f8ee20 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 -- 172.21.15.27:0/801115539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 msgr2=0x559862f912b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbf700cdb80 tx=0x7fbf700d8990 comp rx=0 tx=0).stop 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 -- 172.21.15.27:0/801115539 shutdown_connections 2024-04-26T21:11:29.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f8ee20 0x559862f912b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f8e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 --2- 172.21.15.27:0/801115539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 -- 172.21.15.27:0/801115539 >> 172.21.15.27:0/801115539 conn(0x559862f1dd10 msgr2=0x559862f84d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf8d400640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 0x7fbf6c0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x559862f8a510 tx=0x7fbf80009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 -- 172.21.15.27:0/801115539 shutdown_connections 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.136+0000 7fbf92252740 1 -- 172.21.15.27:0/801115539 wait complete. 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 -- start start 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 Processor -- start 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f93b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f8ee20 0x559862f9abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559862f91e20 con 0x559862f892c0 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559862e74f70 con 0x559862f89cb0 2024-04-26T21:11:29.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559862f91fa0 con 0x559862f8ee20 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.137+0000 7fbf92252740 10 client.0 _ll_register_callbacks cb 0x559862ee3480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54224/0 (socket says 172.21.15.27:54224) 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2493432122 learned_addr learned my addr 172.21.15.27:0/2493432122 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f93b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2493432122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f8ee20 msgr2=0x559862f9abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f8ee20 0x559862f9abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 msgr2=0x559862f93b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f93b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2493432122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559862cc96b0 con 0x559862f892c0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbf88109de0 tx=0x7fbf780043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34decb4c4a6fa325 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf76000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.138+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035030 con 0x559862f892c0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbf78003a00 con 0x559862f892c0 2024-04-26T21:11:29.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf700d24e0 con 0x559862f892c0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf78003c00 con 0x559862f892c0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf92252740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559862f9db70 con 0x559862f892c0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf700d24e0 con 0x559862f892c0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf7803c440 con 0x559862f892c0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.139+0000 7fbf92252740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf92252740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf92252740 20 client.15126 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862fa02e0 0x559862fa1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf92252740 4 client.15126 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 --> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559862efb210 con 0x559862fa02e0 2024-04-26T21:11:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.140+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862fa02e0 0x559862fa1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.141+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862fa02e0 0x559862fa1750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3758c2066eb076 server_cookie=f24aa252baa72fa9 in_seq=0 out_seq=0 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.141+0000 7fbf76000640 10 client.15126 ms_handle_connect on v2:172.21.15.94:6838/12233135 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf76000640 1 -- 172.21.15.27:0/2493432122 <== mds.0 v2:172.21.15.94:6838/12233135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559862efb210 con 0x559862fa02e0 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf76000640 10 client.15126 handle_command_reply: tid=0 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf92252740 1 client.15126 shutdown 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 msgr2=0x559862f93630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbf88109de0 tx=0x7fbf780043b0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 shutdown_connections 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.143+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862fa02e0 0x559862fa1750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f8ee20 0x559862f9abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f93b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 --2- 172.21.15.27:0/2493432122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f892c0 0x559862f93630 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 >> 172.21.15.27:0/2493432122 conn(0x559862f1dd10 msgr2=0x559862f83f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf76000640 0 client.15126 ms_handle_reset on v2:172.21.15.94:6838/12233135 2024-04-26T21:11:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 shutdown_connections 2024-04-26T21:11:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 1 -- 172.21.15.27:0/2493432122 wait complete. 2024-04-26T21:11:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.144+0000 7fbf92252740 20 client.15126 trim_cache size 0 max 16384 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 Processor -- start 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 -- start start 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f1dd10 0x559862f847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f9ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559862f9fa90 con 0x559862f89cb0 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559862fa0160 con 0x559862f1dd10 2024-04-26T21:11:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.164+0000 7fbf92252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559862fa0830 con 0x559862f892c0 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54232/0 (socket says 172.21.15.27:54232) 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 -- 172.21.15.27:0/3118711649 learned_addr learned my addr 172.21.15.27:0/3118711649 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f9ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f1dd10 0x559862f847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 -- 172.21.15.27:0/3118711649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 msgr2=0x559862f9ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f9ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 -- 172.21.15.27:0/3118711649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f1dd10 msgr2=0x559862f847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f1dd10 0x559862f847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.165+0000 7fbf8d400640 1 -- 172.21.15.27:0/3118711649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559862f8a090 con 0x559862f89cb0 2024-04-26T21:11:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf8d400640 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbf800127a0 tx=0x7fbf80014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285a4e0ee00cea07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf7d200640 1 -- 172.21.15.27:0/3118711649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80012e60 con 0x559862f89cb0 2024-04-26T21:11:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf7d200640 1 -- 172.21.15.27:0/3118711649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf8000e850 con 0x559862f89cb0 2024-04-26T21:11:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf7d200640 1 -- 172.21.15.27:0/3118711649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80005480 con 0x559862f89cb0 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf92252740 1 -- 172.21.15.27:0/3118711649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 msgr2=0x559862f83de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.166+0000 7fbf92252740 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbf800127a0 tx=0x7fbf80014070 comp rx=0 tx=0).stop 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 -- 172.21.15.27:0/3118711649 shutdown_connections 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f9ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f89cb0 0x559862f83de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 --2- 172.21.15.27:0/3118711649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f1dd10 0x559862f847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 -- 172.21.15.27:0/3118711649 >> 172.21.15.27:0/3118711649 conn(0x559862f8e920 msgr2=0x559862f836d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 -- 172.21.15.27:0/3118711649 shutdown_connections 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 -- 172.21.15.27:0/3118711649 wait complete. 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 -- start start 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.167+0000 7fbf92252740 1 Processor -- start 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f99e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559862fa15d0 con 0x559862f1dd10 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559862fa0f00 con 0x559862f89cb0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559862fa1ca0 con 0x559862f892c0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf92252740 10 client.0 _ll_register_callbacks cb 0x7fbf78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54240/0 (socket says 172.21.15.27:54240) 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2285759768 learned_addr learned my addr 172.21.15.27:0/2285759768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2285759768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 msgr2=0x559862f96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8d400640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f99e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 msgr2=0x559862f99e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f99e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf8ca00640 1 -- 172.21.15.27:0/2285759768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559862f9a3f0 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.168+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf8ca00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf78006010 tx=0x7fbf78003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c13c883fd7c02a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf76000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78035040 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8000e850 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbf7803cb20 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf7803cd20 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf92252740 10 client.15132 fetch_fsmap learned FSMap version 27 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559863006900 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.169+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf8000e850 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf7804f470 con 0x559862f1dd10 2024-04-26T21:11:29.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 20 client.15132 populate_metadata read hostname 'smithi027' 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862f8ee20 0x559863007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 4 client.15132 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 --> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] -- command(tid 0: {"prefix": "status"}) -- 0x559862ef5080 con 0x559862f8ee20 2024-04-26T21:11:29.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.170+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862f8ee20 0x559863007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.171+0000 7fbf7fe00640 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862f8ee20 0x559863007950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237ee264b0878642 server_cookie=baae3aa59e62b2aa in_seq=0 out_seq=0 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.171+0000 7fbf76000640 10 client.15132 ms_handle_connect on v2:172.21.15.94:6838/12233135 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf76000640 1 -- 172.21.15.27:0/2285759768 <== mds.0 v2:172.21.15.94:6838/12233135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559862ef5080 con 0x559862f8ee20 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf76000640 10 client.15132 handle_command_reply: tid=0 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 client.15132 shutdown 2024-04-26T21:11:29.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 msgr2=0x559862f968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf78006010 tx=0x7fbf78003a30 comp rx=0 tx=0).stop 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 shutdown_connections 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x559862f8ee20 0x559863007950 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559862f892c0 0x559862f96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf76000640 0 client.15132 ms_handle_reset on v2:172.21.15.94:6838/12233135 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559862f89cb0 0x559862f99e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 --2- 172.21.15.27:0/2285759768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559862f1dd10 0x559862f968a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 >> 172.21.15.27:0/2285759768 conn(0x559862f8e920 msgr2=0x559862f88520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.172+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 shutdown_connections 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf92252740 1 -- 172.21.15.27:0/2285759768 wait complete. 2024-04-26T21:11:29.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf92252740 20 client.15132 trim_cache size 0 max 16384 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34517, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 107.391166837, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 113.030221181 2024-04-26T21:11:29.178 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 msgr2=0x7fbf6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 0x7fbf6c0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x559862f8a510 tx=0x7fbf80009580 comp rx=0 tx=0).stop 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 msgr2=0x7fbf8819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000d950 tx=0x7fbf7000de10 comp rx=0 tx=0).stop 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 shutdown_connections 2024-04-26T21:11:29.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fbf6c0a3c40 0x7fbf6c0a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf8810a940 0x7fbf881a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf88109f80 0x7fbf8819f400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 --2- 172.21.15.27:0/2290925294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf88108d90 0x7fbf8819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.173+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 >> 172.21.15.27:0/2290925294 conn(0x7fbf88077d30 msgr2=0x7fbf8810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.174+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 shutdown_connections 2024-04-26T21:11:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:29.174+0000 7fbf8de00640 1 -- 172.21.15.27:0/2290925294 wait complete. 2024-04-26T21:11:29.193 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T21:11:29.193 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:11:29.204 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:11:29.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T21:11:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:29.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:29 smithi027 ceph-mon[28085]: pgmap v423: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:29 smithi204 ceph-mon[32145]: pgmap v423: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:30.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:29 smithi094 ceph-mon[31273]: pgmap v423: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:31.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 -- 172.21.15.27:0/216505791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0109170 msgr2=0x7ff4b010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 --2- 172.21.15.27:0/216505791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0109170 0x7ff4b010b5e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800b0a0 tx=0x7ff4a800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 -- 172.21.15.27:0/216505791 shutdown_connections 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 --2- 172.21.15.27:0/216505791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0109170 0x7ff4b010b5e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 --2- 172.21.15.27:0/216505791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0103f10 0x7ff4b0106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 --2- 172.21.15.27:0/216505791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 0x7ff4b0108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 -- 172.21.15.27:0/216505791 >> 172.21.15.27:0/216505791 conn(0x7ff4b00fd920 msgr2=0x7ff4b00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 -- 172.21.15.27:0/216505791 shutdown_connections 2024-04-26T21:11:31.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.838+0000 7ff4b7600700 1 -- 172.21.15.27:0/216505791 wait complete. 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.839+0000 7ff4b7600700 1 Processor -- start 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.839+0000 7ff4b7600700 1 -- start start 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 0x7ff4b0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 0x7ff4b01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010ee60 con 0x7ff4b0103f10 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010ece0 con 0x7ff4b0109170 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b7600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010efe0 con 0x7ff4b0106840 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:31.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54254/0 (socket says 172.21.15.27:54254) 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b4c00700 1 -- 172.21.15.27:0/3075225778 learned_addr learned my addr 172.21.15.27:0/3075225778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4afe00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 0x7ff4b0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.840+0000 7ff4b5600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 0x7ff4b01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 -- 172.21.15.27:0/3075225778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 msgr2=0x7ff4b0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 0x7ff4b0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 -- 172.21.15.27:0/3075225778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 msgr2=0x7ff4b01a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 0x7ff4b01a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 -- 172.21.15.27:0/3075225778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b01a11d0 con 0x7ff4b0103f10 2024-04-26T21:11:31.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b5600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 0x7ff4b01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b4c00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d980 tx=0x7ff4a400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4016070 con 0x7ff4b0103f10 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a4010460 con 0x7ff4b0103f10 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400f620 con 0x7ff4b0103f10 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.841+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b01a14c0 con 0x7ff4b0103f10 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.842+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b01a1ad0 con 0x7ff4b0103f10 2024-04-26T21:11:31.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.843+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b006a110 con 0x7ff4b0103f10 2024-04-26T21:11:31.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.845+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff4a4019070 con 0x7ff4b0103f10 2024-04-26T21:11:31.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.846+0000 7ff4ad600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 0x7ff4980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:31.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.846+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff4a40d0150 con 0x7ff4b0103f10 2024-04-26T21:11:31.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.847+0000 7ff4afe00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 0x7ff4980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:31.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.847+0000 7ff4afe00700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 0x7ff4980a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:31.851+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4a409c960 con 0x7ff4b0103f10 2024-04-26T21:11:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:31 smithi027 ceph-mon[28085]: pgmap v424: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:32.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:31 smithi204 ceph-mon[32145]: pgmap v424: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:32.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:31 smithi094 ceph-mon[31273]: pgmap v424: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:32.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.107+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff4b0063860 con 0x7ff4980a3d50 2024-04-26T21:11:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.109+0000 7ff4ad600700 1 -- 172.21.15.27:0/3075225778 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff4b0063860 con 0x7ff4980a3d50 2024-04-26T21:11:32.110 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-04-26T21:11:32.110 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-04-26T21:11:32.110 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-04-26T21:11:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 msgr2=0x7ff4980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 0x7ff4980a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).stop 2024-04-26T21:11:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 msgr2=0x7ff4b0199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d980 tx=0x7ff4a400dd40 comp rx=0 tx=0).stop 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 shutdown_connections 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff4980a3d50 0x7ff4980a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b0109170 0x7ff4b01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4b0103f10 0x7ff4b0199570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 --2- 172.21.15.27:0/3075225778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0106840 0x7ff4b0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 >> 172.21.15.27:0/3075225778 conn(0x7ff4b00fd920 msgr2=0x7ff4b0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 shutdown_connections 2024-04-26T21:11:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:32.112+0000 7ff4b7600700 1 -- 172.21.15.27:0/3075225778 wait complete. 2024-04-26T21:11:32.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T21:11:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:33.887 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:33 smithi027 ceph-mon[28085]: pgmap v425: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:33.887 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:33 smithi027 ceph-mon[28085]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:34.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:33 smithi204 ceph-mon[32145]: pgmap v425: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:34.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:33 smithi204 ceph-mon[32145]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:34.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:33 smithi094 ceph-mon[31273]: pgmap v425: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:34.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:33 smithi094 ceph-mon[31273]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 -- 172.21.15.27:0/4170264806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62401068c0 msgr2=0x7f6240106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 --2- 172.21.15.27:0/4170264806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62401068c0 0x7f6240106d20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f622c0099c0 tx=0x7f622c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 -- 172.21.15.27:0/4170264806 shutdown_connections 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 --2- 172.21.15.27:0/4170264806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6240107260 0x7f624010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 --2- 172.21.15.27:0/4170264806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401056c0 0x7f6240105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 --2- 172.21.15.27:0/4170264806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62401068c0 0x7f6240106d20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 -- 172.21.15.27:0/4170264806 >> 172.21.15.27:0/4170264806 conn(0x7f6240100c80 msgr2=0x7f62401030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 -- 172.21.15.27:0/4170264806 shutdown_connections 2024-04-26T21:11:35.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.831+0000 7f6245800700 1 -- 172.21.15.27:0/4170264806 wait complete. 2024-04-26T21:11:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 Processor -- start 2024-04-26T21:11:35.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 -- start start 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 0x7f624019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401068c0 0x7f624019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62401110b0 con 0x7f6240107260 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6240110f30 con 0x7f62401068c0 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.833+0000 7f6245800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6240111230 con 0x7f62401056c0 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:35.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54268/0 (socket says 172.21.15.27:54268) 2024-04-26T21:11:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 -- 172.21.15.27:0/2810999992 learned_addr learned my addr 172.21.15.27:0/2810999992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623ea00700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 0x7f624019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623e000700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401068c0 0x7f624019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:35.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 -- 172.21.15.27:0/2810999992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 msgr2=0x7f624019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 0x7f624019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 -- 172.21.15.27:0/2810999992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401068c0 msgr2=0x7f624019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401068c0 0x7f624019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623f400700 1 -- 172.21.15.27:0/2810999992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401a33e0 con 0x7f6240107260 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.834+0000 7f623ea00700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 0x7f624019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f623f400700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f623800b9a0 tx=0x7f623800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238019070 con 0x7f6240107260 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f623800ce40 con 0x7f6240107260 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401a36d0 con 0x7f6240107260 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238007370 con 0x7f6240107260 2024-04-26T21:11:35.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.835+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401a3c10 con 0x7f6240107260 2024-04-26T21:11:35.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.836+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624006a110 con 0x7f6240107260 2024-04-26T21:11:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.838+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f623801c070 con 0x7f6240107260 2024-04-26T21:11:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.840+0000 7f6237400700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 0x7f62240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.840+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f62380cf940 con 0x7f6240107260 2024-04-26T21:11:35.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.840+0000 7f623ea00700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 0x7f62240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:35.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.841+0000 7f623ea00700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 0x7f62240a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6230005950 tx=0x7f62300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:35.845+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f623809c380 con 0x7f6240107260 2024-04-26T21:11:35.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:35 smithi027 ceph-mon[28085]: pgmap v426: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:36.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:35 smithi204 ceph-mon[32145]: pgmap v426: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:36.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:35 smithi094 ceph-mon[31273]: pgmap v426: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:36.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.103+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6240063860 con 0x7f62240a3d50 2024-04-26T21:11:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.120+0000 7f6237400700 1 -- 172.21.15.27:0/2810999992 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f6240063860 con 0x7f62240a3d50 2024-04-26T21:11:36.122 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T21:11:36.122 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 24s ago 2m 16.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9f9b1def5d51 2024-04-26T21:11:36.122 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi094 running (2m) 47s ago 119s 24.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c8825ae95623 2024-04-26T21:11:36.122 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi204 running (2m) 48s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 63a3836eb533 2024-04-26T21:11:36.122 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (2m) 24s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fa27592029f2 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi094 running (2m) 47s ago 2m 16.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 f80392088d2f 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi204 running (2m) 48s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 825fa59664a4 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (2m) 24s ago 2m 16.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 2bb0e60da5ec 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi094 running (2m) 47s ago 2m 16.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 121b01ab36cc 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi204 running (2m) 48s ago 2m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 b153d31e5de2 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (2m) 24s ago 2m 12.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9333d71dbf2d 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi094 running (2m) 47s ago 2m 16.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9ecf582fb888 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi204 running (2m) 48s ago 2m 15.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4e50074c1ca0 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (12m) 24s ago 12m 517M - 19.0.0-3306-gbc612ffb 55dc54019ad8 eb89e39ef638 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi094 *:8443,8765 running (9m) 47s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c7dbc05a7ad5 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi204 *:8443,8765 running (9m) 48s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 879f6c2ad24e 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (12m) 24s ago 12m 50.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 0e6c7ddb8f6e 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi094 running (10m) 47s ago 10m 39.7M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 db99f4544d22 2024-04-26T21:11:36.123 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi204 running (10m) 48s ago 10m 42.8M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 170f35147ffe 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (9m) 24s ago 9m 54.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 d757af214d39 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi094 running (8m) 47s ago 8m 48.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 69db734a0b9a 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi204 running (8m) 48s ago 8m 49.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b2fe0c47cd3f 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 24s ago 7m 53.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 31241fb78cda 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi094 running (7m) 47s ago 7m 44.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 a0150fd4f3d8 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi204 running (6m) 48s ago 6m 52.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4ef1861eb721 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (6m) 24s ago 6m 51.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 89a6762de6c1 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi094 running (5m) 47s ago 5m 45.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 12274264afb7 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi204 running (4m) 48s ago 4m 52.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 96c99154793e 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 24s ago 4m 50.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 0936803af54a 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi094 running (3m) 47s ago 3m 45.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 8f96df0dc6a3 2024-04-26T21:11:36.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi204 running (3m) 48s ago 3m 50.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4cb0fe0e018e 2024-04-26T21:11:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.124+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 msgr2=0x7f62240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.124+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 0x7f62240a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6230005950 tx=0x7f62300058e0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.124+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 msgr2=0x7f62401a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.124+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f623800b9a0 tx=0x7f623800bd60 comp rx=0 tx=0).stop 2024-04-26T21:11:36.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 shutdown_connections 2024-04-26T21:11:36.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f62240a3d50 0x7f62240a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6240107260 0x7f62401a2d40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62401056c0 0x7f624019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 --2- 172.21.15.27:0/2810999992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62401068c0 0x7f624019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 >> 172.21.15.27:0/2810999992 conn(0x7f6240100c80 msgr2=0x7f6240109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 shutdown_connections 2024-04-26T21:11:36.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:36.125+0000 7f6245800700 1 -- 172.21.15.27:0/2810999992 wait complete. 2024-04-26T21:11:36.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T21:11:37.458 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:37.908 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:37 smithi027 ceph-mon[28085]: pgmap v427: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:37.908 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:37 smithi027 ceph-mon[28085]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:38.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:37 smithi204 ceph-mon[32145]: pgmap v427: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:38.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:37 smithi204 ceph-mon[32145]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:38.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:37 smithi094 ceph-mon[31273]: pgmap v427: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:38.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:37 smithi094 ceph-mon[31273]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 -- 172.21.15.27:0/264107587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9940107260 msgr2=0x7f994010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 --2- 172.21.15.27:0/264107587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9940107260 0x7f994010db10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f993800b0d0 tx=0x7f993800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 -- 172.21.15.27:0/264107587 shutdown_connections 2024-04-26T21:11:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 --2- 172.21.15.27:0/264107587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9940107260 0x7f994010db10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 --2- 172.21.15.27:0/264107587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 0x7f9940105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 --2- 172.21.15.27:0/264107587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99401068c0 0x7f9940106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 -- 172.21.15.27:0/264107587 >> 172.21.15.27:0/264107587 conn(0x7f9940100c80 msgr2=0x7f99401030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 -- 172.21.15.27:0/264107587 shutdown_connections 2024-04-26T21:11:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.541+0000 7f9948600700 1 -- 172.21.15.27:0/264107587 wait complete. 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.543+0000 7f9948600700 1 Processor -- start 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.543+0000 7f9948600700 1 -- start start 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 0x7f994019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9940107260 0x7f99401a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9940111100 con 0x7f99401068c0 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9940110f80 con 0x7f99401056c0 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9948600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9940111280 con 0x7f9940107260 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9945200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9945200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38464/0 (socket says 172.21.15.27:38464) 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9945200700 1 -- 172.21.15.27:0/1794557248 learned_addr learned my addr 172.21.15.27:0/1794557248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:39.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9945c00700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 0x7f994019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.544+0000 7f9946600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9940107260 0x7f99401a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 -- 172.21.15.27:0/1794557248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9940107260 msgr2=0x7f99401a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9940107260 0x7f99401a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 -- 172.21.15.27:0/1794557248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 msgr2=0x7f994019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 0x7f994019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 -- 172.21.15.27:0/1794557248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99401a3440 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9945200700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b560 tx=0x7f992c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c015020 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f992c011cc0 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99401a3730 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.545+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c010d30 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.546+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a3cf0 con 0x7f99401068c0 2024-04-26T21:11:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.547+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994006a110 con 0x7f99401068c0 2024-04-26T21:11:39.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.549+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f992c018020 con 0x7f99401068c0 2024-04-26T21:11:39.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.551+0000 7f993e000700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 0x7f99280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.551+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f992c0cfc40 con 0x7f99401068c0 2024-04-26T21:11:39.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.551+0000 7f9945c00700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 0x7f99280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.552+0000 7f9945c00700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 0x7f99280a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9940106720 tx=0x7f9934009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:39.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.555+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f992c01f020 con 0x7f99401068c0 2024-04-26T21:11:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.800+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9940063860 con 0x7f99280a3cd0 2024-04-26T21:11:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.818+0000 7f993e000700 1 -- 172.21.15.27:0/1794557248 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9940063860 con 0x7f99280a3cd0 2024-04-26T21:11:39.819 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T21:11:39.819 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 51s ago 2m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-04-26T21:11:39.820 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 51s ago 10m smithi027=x;smithi094=y;smithi204=z;count:3 2024-04-26T21:11:39.820 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 51s ago 10m smithi027:172.21.15.27=a;smithi094:172.21.15.94=b;smithi204:172.21.15.204=c;count:3 2024-04-26T21:11:39.820 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 51s ago - 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 msgr2=0x7f99280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 0x7f99280a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9940106720 tx=0x7f9934009190 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 msgr2=0x7f994019bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b560 tx=0x7f992c00b870 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 shutdown_connections 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f99280a3cd0 0x7f99280a6190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9940107260 0x7f99401a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99401056c0 0x7f994019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 --2- 172.21.15.27:0/1794557248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99401068c0 0x7f994019bdc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.821+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 >> 172.21.15.27:0/1794557248 conn(0x7f9940100c80 msgr2=0x7f9940109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.822+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 shutdown_connections 2024-04-26T21:11:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:39.822+0000 7f9948600700 1 -- 172.21.15.27:0/1794557248 wait complete. 2024-04-26T21:11:40.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:39 smithi204 ceph-mon[32145]: pgmap v428: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:40.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:39 smithi094 ceph-mon[31273]: pgmap v428: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:40.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:39 smithi027 ceph-mon[28085]: pgmap v428: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:40.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T21:11:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:41.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:41 smithi027 ceph-mon[28085]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:41.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:41 smithi027 ceph-mon[28085]: pgmap v429: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:42.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:41 smithi204 ceph-mon[32145]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:42.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:41 smithi204 ceph-mon[32145]: pgmap v429: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:42.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:41 smithi094 ceph-mon[31273]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:42.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:41 smithi094 ceph-mon[31273]: pgmap v429: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 -- 172.21.15.27:0/926068324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410abb0 msgr2=0x7ff3b410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 --2- 172.21.15.27:0/926068324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410abb0 0x7ff3b410b010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0099c0 tx=0x7ff39c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 -- 172.21.15.27:0/926068324 shutdown_connections 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 --2- 172.21.15.27:0/926068324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410b550 0x7ff3b4111e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 --2- 172.21.15.27:0/926068324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 0x7ff3b4109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 --2- 172.21.15.27:0/926068324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410abb0 0x7ff3b410b010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 -- 172.21.15.27:0/926068324 >> 172.21.15.27:0/926068324 conn(0x7ff3b4078980 msgr2=0x7ff3b407adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 -- 172.21.15.27:0/926068324 shutdown_connections 2024-04-26T21:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.446+0000 7ff3b3200700 1 -- 172.21.15.27:0/926068324 wait complete. 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.447+0000 7ff3b3200700 1 Processor -- start 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.447+0000 7ff3b3200700 1 -- start start 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 0x7ff3b419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410abb0 0x7ff3b419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4115350 con 0x7ff3b410b550 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b41151d0 con 0x7ff3b41099b0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b3200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b41154d0 con 0x7ff3b410abb0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38490/0 (socket says 172.21.15.27:38490) 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.448+0000 7ff3b2800700 1 -- 172.21.15.27:0/3286050540 learned_addr learned my addr 172.21.15.27:0/3286050540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b1e00700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 0x7ff3b419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3abe00700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410abb0 0x7ff3b419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 -- 172.21.15.27:0/3286050540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410abb0 msgr2=0x7ff3b419fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410abb0 0x7ff3b419fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 -- 172.21.15.27:0/3286050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 msgr2=0x7ff3b419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 0x7ff3b419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 -- 172.21.15.27:0/3286050540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b41a75f0 con 0x7ff3b410b550 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.449+0000 7ff3b2800700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00add0 tx=0x7ff3ac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.450+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac017020 con 0x7ff3b410b550 2024-04-26T21:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.450+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b41a78e0 con 0x7ff3b410b550 2024-04-26T21:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.450+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3ac00ae00 con 0x7ff3b410b550 2024-04-26T21:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.450+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b41a7e20 con 0x7ff3b410b550 2024-04-26T21:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.450+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0052f0 con 0x7ff3b410b550 2024-04-26T21:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.451+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b406a110 con 0x7ff3b410b550 2024-04-26T21:11:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.453+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3ac004ad0 con 0x7ff3b410b550 2024-04-26T21:11:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.454+0000 7ff3aaa00700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 0x7ff3980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.455+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff3ac0cfd00 con 0x7ff3b410b550 2024-04-26T21:11:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.455+0000 7ff3b1e00700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 0x7ff3980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.456+0000 7ff3b1e00700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 0x7ff3980a6190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff3b410aa10 tx=0x7ff3a400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.459+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff3ac021020 con 0x7ff3b410b550 2024-04-26T21:11:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.696+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff3b4063860 con 0x7ff3980a3cd0 2024-04-26T21:11:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.699+0000 7ff3aaa00700 1 -- 172.21.15.27:0/3286050540 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7ff3b4063860 con 0x7ff3980a3cd0 2024-04-26T21:11:43.700 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-04-26T21:11:43.700 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-04-26T21:11:43.701 INFO:teuthology.orchestra.run.smithi027.stdout:smithi094 172.21.15.94 2024-04-26T21:11:43.701 INFO:teuthology.orchestra.run.smithi027.stdout:smithi204 172.21.15.204 2024-04-26T21:11:43.701 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 msgr2=0x7ff3980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 0x7ff3980a6190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff3b410aa10 tx=0x7ff3a400a3b0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 msgr2=0x7ff3b41a6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00add0 tx=0x7ff3ac00c330 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 shutdown_connections 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7ff3980a3cd0 0x7ff3980a6190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3b410b550 0x7ff3b41a6f50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b41099b0 0x7ff3b419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 --2- 172.21.15.27:0/3286050540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b410abb0 0x7ff3b419fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.702+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 >> 172.21.15.27:0/3286050540 conn(0x7ff3b4078980 msgr2=0x7ff3b410df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.703+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 shutdown_connections 2024-04-26T21:11:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:43.703+0000 7ff3b3200700 1 -- 172.21.15.27:0/3286050540 wait complete. 2024-04-26T21:11:43.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:43 smithi027 ceph-mon[28085]: pgmap v430: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:44.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:43 smithi204 ceph-mon[32145]: pgmap v430: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:44.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:43 smithi094 ceph-mon[31273]: pgmap v430: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-26T21:11:44.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T21:11:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:45.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:44 smithi094 ceph-mon[31273]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:45.106 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:44 smithi027 ceph-mon[28085]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:45.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:44 smithi204 ceph-mon[32145]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:46.019 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:45 smithi027 ceph-mon[28085]: pgmap v431: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:46.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:45 smithi094 ceph-mon[31273]: pgmap v431: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:45 smithi204 ceph-mon[32145]: pgmap v431: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 -- 172.21.15.27:0/749273378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 msgr2=0x7f974410db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 --2- 172.21.15.27:0/749273378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f974410db10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f973c00b0d0 tx=0x7f973c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 -- 172.21.15.27:0/749273378 shutdown_connections 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 --2- 172.21.15.27:0/749273378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f974410db10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 --2- 172.21.15.27:0/749273378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441056c0 0x7f9744105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 --2- 172.21.15.27:0/749273378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441068c0 0x7f9744106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.214+0000 7f974aa00700 1 -- 172.21.15.27:0/749273378 >> 172.21.15.27:0/749273378 conn(0x7f9744100c80 msgr2=0x7f97441030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.215+0000 7f974aa00700 1 -- 172.21.15.27:0/749273378 shutdown_connections 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.215+0000 7f974aa00700 1 -- 172.21.15.27:0/749273378 wait complete. 2024-04-26T21:11:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 Processor -- start 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 -- start start 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441056c0 0x7f974419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441068c0 0x7f974419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9744111060 con 0x7f9744107260 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9744110ee0 con 0x7f97441056c0 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.216+0000 7f974aa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97441111e0 con 0x7f97441068c0 2024-04-26T21:11:47.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38520/0 (socket says 172.21.15.27:38520) 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 -- 172.21.15.27:0/1583919356 learned_addr learned my addr 172.21.15.27:0/1583919356 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9743e00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441056c0 0x7f974419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9743400700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441068c0 0x7f974419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 -- 172.21.15.27:0/1583919356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441068c0 msgr2=0x7f974419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441068c0 0x7f974419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 -- 172.21.15.27:0/1583919356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441056c0 msgr2=0x7f974419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441056c0 0x7f974419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.217+0000 7f9748a00700 1 -- 172.21.15.27:0/1583919356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97441a3300 con 0x7f9744107260 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f9748a00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f973c0095e0 tx=0x7f973c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c01f030 con 0x7f9744107260 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f973c007a00 con 0x7f9744107260 2024-04-26T21:11:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97441a3590 con 0x7f9744107260 2024-04-26T21:11:47.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c01b690 con 0x7f9744107260 2024-04-26T21:11:47.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.218+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441a3a90 con 0x7f9744107260 2024-04-26T21:11:47.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.219+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974406a110 con 0x7f9744107260 2024-04-26T21:11:47.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.221+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f973c022030 con 0x7f9744107260 2024-04-26T21:11:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.223+0000 7f9740c00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 0x7f97280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.223+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f973c00f020 con 0x7f9744107260 2024-04-26T21:11:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.223+0000 7f9743e00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 0x7f97280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.224+0000 7f9743e00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 0x7f97280a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9734005950 tx=0x7f97340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.227+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f973c029020 con 0x7f9744107260 2024-04-26T21:11:47.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.466+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 --> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9744063920 con 0x7f97280a3d50 2024-04-26T21:11:47.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.469+0000 7f9740c00700 1 -- 172.21.15.27:0/1583919356 <== mgr.14152 v2:172.21.15.27:6800/3365757774 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9744063920 con 0x7f97280a3d50 2024-04-26T21:11:47.475 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T21:11:47.475 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T21:11:47.475 INFO:teuthology.orchestra.run.smithi027.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:11:47.476 INFO:teuthology.orchestra.run.smithi027.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:11:47.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 msgr2=0x7f97280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 0x7f97280a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9734005950 tx=0x7f97340058e0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 msgr2=0x7f97441a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f973c0095e0 tx=0x7f973c009c30 comp rx=0 tx=0).stop 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 shutdown_connections 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f97280a3d50 0x7f97280a6210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744107260 0x7f97441a2c60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97441056c0 0x7f974419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 --2- 172.21.15.27:0/1583919356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97441068c0 0x7f974419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.472+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 >> 172.21.15.27:0/1583919356 conn(0x7f9744100c80 msgr2=0x7f9744109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.473+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 shutdown_connections 2024-04-26T21:11:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:47.473+0000 7f974aa00700 1 -- 172.21.15.27:0/1583919356 wait complete. 2024-04-26T21:11:48.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:47 smithi094 ceph-mon[31273]: pgmap v432: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:48.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:47 smithi027 ceph-mon[28085]: pgmap v432: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:47 smithi204 ceph-mon[32145]: pgmap v432: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:11:48.264 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:11:48.274 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:11:48.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T21:11:48.748 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:49.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:48 smithi027 ceph-mon[28085]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:48 smithi204 ceph-mon[32145]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:48 smithi094 ceph-mon[31273]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:11:49.944 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:49 smithi027 ceph-mon[28085]: pgmap v433: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:50.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:49 smithi204 ceph-mon[32145]: pgmap v433: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:50.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:49 smithi094 ceph-mon[31273]: pgmap v433: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:11:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 -- 172.21.15.27:0/1714209328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 msgr2=0x7f44dc10b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 --2- 172.21.15.27:0/1714209328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc10b920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b0d0 tx=0x7f44d400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 -- 172.21.15.27:0/1714209328 shutdown_connections 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 --2- 172.21.15.27:0/1714209328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc10b920 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 --2- 172.21.15.27:0/1714209328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1034d0 0x7f44dc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 --2- 172.21.15.27:0/1714209328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1046d0 0x7f44dc104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.982+0000 7f44e1600700 1 -- 172.21.15.27:0/1714209328 >> 172.21.15.27:0/1714209328 conn(0x7f44dc0fea70 msgr2=0x7f44dc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.983+0000 7f44e1600700 1 -- 172.21.15.27:0/1714209328 shutdown_connections 2024-04-26T21:11:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.983+0000 7f44e1600700 1 -- 172.21.15.27:0/1714209328 wait complete. 2024-04-26T21:11:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 Processor -- start 2024-04-26T21:11:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 -- start start 2024-04-26T21:11:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 0x7f44dc19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1046d0 0x7f44dc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10ee50 con 0x7f44dc105070 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10ecd0 con 0x7f44dc1046d0 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.984+0000 7f44e1600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10efd0 con 0x7f44dc1034d0 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57628/0 (socket says 172.21.15.27:57628) 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 -- 172.21.15.27:0/700842955 learned_addr learned my addr 172.21.15.27:0/700842955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44daa00700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 0x7f44dc19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44d3e00700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1046d0 0x7f44dc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 -- 172.21.15.27:0/700842955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 msgr2=0x7f44dc19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 0x7f44dc19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 -- 172.21.15.27:0/700842955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1046d0 msgr2=0x7f44dc19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1046d0 0x7f44dc19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44db400700 1 -- 172.21.15.27:0/700842955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc1a3300 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.985+0000 7f44daa00700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 0x7f44dc19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44db400700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f44d40095e0 tx=0x7f44d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d401f030 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc1a3590 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44d4007a00 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d401b690 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.986+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1a3a40 con 0x7f44dc105070 2024-04-26T21:11:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.987+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc06a110 con 0x7f44dc105070 2024-04-26T21:11:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.989+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f44d4022030 con 0x7f44dc105070 2024-04-26T21:11:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.991+0000 7f44d3400700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 0x7f44c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.991+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f44d40d0ff0 con 0x7f44dc105070 2024-04-26T21:11:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.991+0000 7f44daa00700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 0x7f44c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.992+0000 7f44daa00700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 0x7f44c00a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005950 tx=0x7f44cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:50.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:50.994+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f44d409d8d0 con 0x7f44dc105070 2024-04-26T21:11:51.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:50 smithi027 ceph-mon[28085]: pgmap v434: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:51.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:50 smithi204 ceph-mon[32145]: pgmap v434: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:51.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.274+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f44dc1a3e60 con 0x7f44dc105070 2024-04-26T21:11:51.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:50 smithi094 ceph-mon[31273]: pgmap v434: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:11:51.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.275+0000 7f44d3400700 1 -- 172.21.15.27:0/700842955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3082 (secure 0 0 0) 0x7f44d409d1d0 con 0x7f44dc105070 2024-04-26T21:11:51.277 INFO:teuthology.orchestra.run.smithi027.stdout:e27 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 27 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-04-26T21:09:40.863524+0000 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-04-26T21:09:49.487318+0000 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-04-26T21:11:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 5 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2,3,4 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=34517,1=34472,2=34430,3=34406,4=14640} 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-04-26T21:11:51.279 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 34517 members: 34472,34430,34517,34406,14640 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:34517} state up:active seq 3 addr [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{0:24505} state up:standby-replay seq 1 addr [v2:172.21.15.204:6838/1122553339,v1:172.21.15.204:6839/1122553339] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{1:34472} state up:active seq 8 addr [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/743097112,v1:172.21.15.204:6837/743097112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{2:34430} state up:active seq 11 addr [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{2:24463} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/1241203213,v1:172.21.15.204:6835/1241203213] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{3:34406} state up:active seq 14 addr [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{3:24421} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/3519578339,v1:172.21.15.204:6833/3519578339] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{4:14640} state up:active seq 6 addr [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{4:14622} state up:standby-replay seq 1 addr [v2:172.21.15.27:6838/4118119230,v1:172.21.15.27:6839/4118119230] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.27:6834/2990870611,v1:172.21.15.27:6835/2990870611] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.281 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.27:6836/1830505768,v1:172.21.15.27:6837/1830505768] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.278+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 msgr2=0x7f44c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.278+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 0x7f44c00a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005950 tx=0x7f44cc0058e0 comp rx=0 tx=0).stop 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.278+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 msgr2=0x7f44dc1a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.278+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f44d40095e0 tx=0x7f44d4009c30 comp rx=0 tx=0).stop 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 shutdown_connections 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f44c00a3d50 0x7f44c00a6210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:51.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44dc105070 0x7f44dc1a2c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44dc1034d0 0x7f44dc19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 --2- 172.21.15.27:0/700842955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1046d0 0x7f44dc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 >> 172.21.15.27:0/700842955 conn(0x7f44dc0fea70 msgr2=0x7f44dc075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 shutdown_connections 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:51.279+0000 7f44e1600700 1 -- 172.21.15.27:0/700842955 wait complete. 2024-04-26T21:11:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:11:52.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:51 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/700842955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:11:52.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a125c556-040f-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T21:11:52.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:51 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/700842955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:11:52.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:51 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/700842955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:11:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:11:53.067 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:52 smithi027 ceph-mon[28085]: pgmap v435: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:53.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:52 smithi204 ceph-mon[32145]: pgmap v435: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:52 smithi094 ceph-mon[31273]: pgmap v435: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:11:54.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 -- 172.21.15.27:0/3896942910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401056c0 msgr2=0x7f3040105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 --2- 172.21.15.27:0/3896942910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401056c0 0x7f3040105ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30300099c0 tx=0x7f3030009cd0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 -- 172.21.15.27:0/3896942910 shutdown_connections 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 --2- 172.21.15.27:0/3896942910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 0x7f304010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 --2- 172.21.15.27:0/3896942910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401056c0 0x7f3040105ae0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 --2- 172.21.15.27:0/3896942910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401068c0 0x7f3040106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 -- 172.21.15.27:0/3896942910 >> 172.21.15.27:0/3896942910 conn(0x7f3040100c80 msgr2=0x7f30401030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 -- 172.21.15.27:0/3896942910 shutdown_connections 2024-04-26T21:11:54.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.987+0000 7f3045c00700 1 -- 172.21.15.27:0/3896942910 wait complete. 2024-04-26T21:11:54.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 Processor -- start 2024-04-26T21:11:54.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 -- start start 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 0x7f304019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 0x7f30401a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3040111060 con 0x7f30401068c0 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3040110ee0 con 0x7f30401056c0 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.989+0000 7f3045c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30401111e0 con 0x7f3040107260 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57632/0 (socket says 172.21.15.27:57632) 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 -- 172.21.15.27:0/4287483786 learned_addr learned my addr 172.21.15.27:0/4287483786 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303ea00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 0x7f304019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303f400700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 0x7f30401a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 -- 172.21.15.27:0/4287483786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 msgr2=0x7f30401a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 0x7f30401a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 -- 172.21.15.27:0/4287483786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 msgr2=0x7f304019b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 0x7f304019b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303e000700 1 -- 172.21.15.27:0/4287483786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30401a3390 con 0x7f30401068c0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.990+0000 7f303ea00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 0x7f304019b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f303e000700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f302c00d9b0 tx=0x7f302c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c016070 con 0x7f30401068c0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f302c004020 con 0x7f30401068c0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c004bb0 con 0x7f30401068c0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30401a3680 con 0x7f30401068c0 2024-04-26T21:11:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.991+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30401a3b70 con 0x7f30401068c0 2024-04-26T21:11:54.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.993+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304006a110 con 0x7f30401068c0 2024-04-26T21:11:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.995+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f302c00bb60 con 0x7f30401068c0 2024-04-26T21:11:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.997+0000 7f3037400700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 0x7f30240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.997+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f302c0d0450 con 0x7f30401068c0 2024-04-26T21:11:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.997+0000 7f303ea00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 0x7f30240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:54.998+0000 7f303ea00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 0x7f30240a6210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30300098e0 tx=0x7f303001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.002+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f302c01f020 con 0x7f30401068c0 2024-04-26T21:11:55.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.236+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3040064030 con 0x7f30401068c0 2024-04-26T21:11:55.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.237+0000 7f3037400700 1 -- 172.21.15.27:0/4287483786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5298 (secure 0 0 0) 0x7f302c00f3c0 con 0x7f30401068c0 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 90 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:fsid a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-04-26T20:58:51.280981+0000 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-04-26T21:10:03.670913+0000 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 26 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-04-26T21:11:55.240 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/530549701,v1:172.21.15.27:6803/530549701] [v2:172.21.15.27:6804/530549701,v1:172.21.15.27:6805/530549701] exists,up fa98cfe0-26a3-4f94-bc02-6c01411e6ec3 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3294273778,v1:172.21.15.94:6801/3294273778] [v2:172.21.15.94:6802/3294273778,v1:172.21.15.94:6803/3294273778] exists,up b88f3050-7a38-4f9e-8719-47deef44eb6f 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3559310933,v1:172.21.15.204:6801/3559310933] [v2:172.21.15.204:6802/3559310933,v1:172.21.15.204:6803/3559310933] exists,up 689b4746-40ea-4411-bdb5-2aeedd58b38e 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/2585963482,v1:172.21.15.27:6811/2585963482] [v2:172.21.15.27:6812/2585963482,v1:172.21.15.27:6813/2585963482] exists,up 496145b1-6477-4d0e-a222-c6222e95eac0 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/128441754,v1:172.21.15.94:6809/128441754] [v2:172.21.15.94:6810/128441754,v1:172.21.15.94:6811/128441754] exists,up c62db123-2dc9-4592-ad16-f2fe0723e9cd 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/4134827726,v1:172.21.15.204:6809/4134827726] [v2:172.21.15.204:6810/4134827726,v1:172.21.15.204:6811/4134827726] exists,up 380e7876-c897-462a-96ee-0f6ef1239fb5 2024-04-26T21:11:55.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 48 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/1091396972,v1:172.21.15.27:6819/1091396972] [v2:172.21.15.27:6820/1091396972,v1:172.21.15.27:6821/1091396972] exists,up a2c9acfd-f6a3-41e6-a3d3-b694781bd208 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 53 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/1853319664,v1:172.21.15.94:6817/1853319664] [v2:172.21.15.94:6818/1853319664,v1:172.21.15.94:6819/1853319664] exists,up f1776a60-ed45-4ece-8668-fe147a04409c 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/3522528239,v1:172.21.15.204:6817/3522528239] [v2:172.21.15.204:6818/3522528239,v1:172.21.15.204:6819/3522528239] exists,up 3c87f761-8ce7-4634-92e9-223f67e64af6 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 70 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/2596688206,v1:172.21.15.27:6827/2596688206] [v2:172.21.15.27:6828/2596688206,v1:172.21.15.27:6829/2596688206] exists,up 53a783f5-a861-4ea2-b77f-f54e79b19da2 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 75 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/141255194,v1:172.21.15.94:6825/141255194] [v2:172.21.15.94:6826/141255194,v1:172.21.15.94:6827/141255194] exists,up 70efbe0c-9374-43d6-b479-f1412debf3e7 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/1542928815,v1:172.21.15.204:6825/1542928815] [v2:172.21.15.204:6826/1542928815,v1:172.21.15.204:6827/1542928815] exists,up c543ea6d-1e49-42e8-aea5-1cea36e42415 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.b [8,11] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.14 [2,4] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.16 [2,4] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1f [8,1,7,11] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.c [3,9] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1b [4,6] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1d [5,9] 2024-04-26T21:11:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.3a [4,8] 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.3c [4,9] 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2516445598 expires 2024-04-27T21:00:01.870639+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2901658935 expires 2024-04-27T21:00:01.870639+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/2094082344 expires 2024-04-27T20:59:25.698004+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2239698495 expires 2024-04-27T20:59:25.698004+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/128242469 expires 2024-04-27T21:00:01.870639+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/2094082344 expires 2024-04-27T20:59:25.698004+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3447033143 expires 2024-04-27T21:00:01.870639+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2522153056 expires 2024-04-27T20:59:25.698004+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/128242469 expires 2024-04-27T21:00:01.870639+0000 2024-04-26T21:11:55.243 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1042211156 expires 2024-04-27T20:59:25.698004+0000 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.240+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 msgr2=0x7f30240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.240+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 0x7f30240a6210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30300098e0 tx=0x7f303001a040 comp rx=0 tx=0).stop 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.240+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 msgr2=0x7f304019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.240+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f302c00d9b0 tx=0x7f302c00dd70 comp rx=0 tx=0).stop 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 shutdown_connections 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f30240a3d50 0x7f30240a6210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3040107260 0x7f30401a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30401056c0 0x7f304019b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 --2- 172.21.15.27:0/4287483786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30401068c0 0x7f304019bc70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 >> 172.21.15.27:0/4287483786 conn(0x7f3040100c80 msgr2=0x7f3040109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 shutdown_connections 2024-04-26T21:11:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:55.241+0000 7f3045c00700 1 -- 172.21.15.27:0/4287483786 wait complete. 2024-04-26T21:11:55.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:55 smithi027 ceph-mon[28085]: pgmap v436: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:55.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:55 smithi204 ceph-mon[32145]: pgmap v436: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:55.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:55 smithi094 ceph-mon[31273]: pgmap v436: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:11:56.037 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T21:11:56.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:56.234 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:56 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4287483786' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:11:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 -- 172.21.15.27:0/2487491230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298108d90 msgr2=0x7f62981091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 --2- 172.21.15.27:0/2487491230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298108d90 0x7f62981091b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6288009900 tx=0x7f628802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 -- 172.21.15.27:0/2487491230 shutdown_connections 2024-04-26T21:11:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 --2- 172.21.15.27:0/2487491230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629810a940 0x7f62981111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 --2- 172.21.15.27:0/2487491230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6298109f80 0x7f629810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 --2- 172.21.15.27:0/2487491230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298108d90 0x7f62981091b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 -- 172.21.15.27:0/2487491230 >> 172.21.15.27:0/2487491230 conn(0x7f6298077d30 msgr2=0x7f629807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 -- 172.21.15.27:0/2487491230 shutdown_connections 2024-04-26T21:11:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.273+0000 7f629d200640 1 -- 172.21.15.27:0/2487491230 wait complete. 2024-04-26T21:11:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.274+0000 7f629d200640 1 Processor -- start 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.274+0000 7f629d200640 1 -- start start 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 0x7f629819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629810a940 0x7f62981a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62981147c0 con 0x7f6298109f80 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114640 con 0x7f629810a940 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f629d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114940 con 0x7f6298108d90 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f6296a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f6296a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57654/0 (socket says 172.21.15.27:57654) 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f6296a00640 1 -- 172.21.15.27:0/1648918633 learned_addr learned my addr 172.21.15.27:0/1648918633 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f6297400640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 0x7f629819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.275+0000 7f6297e00640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629810a940 0x7f62981a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 -- 172.21.15.27:0/1648918633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 msgr2=0x7f629819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 0x7f629819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 -- 172.21.15.27:0/1648918633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629810a940 msgr2=0x7f62981a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629810a940 0x7f62981a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 -- 172.21.15.27:0/1648918633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62981a6af0 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6297400640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 0x7f629819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f6296a00640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f628400d950 tx=0x7f628400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284017030 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6284004020 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62981a6de0 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.276+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62840055d0 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.277+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a72f0 con 0x7f6298109f80 2024-04-26T21:11:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.278+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6298110180 con 0x7f6298109f80 2024-04-26T21:11:56.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.281+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6284009a30 con 0x7f6298109f80 2024-04-26T21:11:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.282+0000 7f628fe00640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 0x7f627c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.283+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f62840d0070 con 0x7f6298109f80 2024-04-26T21:11:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.283+0000 7f6297400640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 0x7f627c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.284+0000 7f6297400640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 0x7f627c0a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62880057c0 tx=0x7f6288031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:56.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.287+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f628409c880 con 0x7f6298109f80 2024-04-26T21:11:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:56 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4287483786' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:11:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.501+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f629802dcf0 con 0x7f6298109f80 2024-04-26T21:11:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.506+0000 7f628fe00640 1 -- 172.21.15.27:0/1648918633 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7f628409c180 con 0x7f6298109f80 2024-04-26T21:11:56.508 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:56.508 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 msgr2=0x7f627c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 0x7f627c0a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62880057c0 tx=0x7f6288031400 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 msgr2=0x7f629819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f628400d950 tx=0x7f628400de10 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 shutdown_connections 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f627c0a3cc0 0x7f627c0a6180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f629810a940 0x7f62981a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6298109f80 0x7f629819f4b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 --2- 172.21.15.27:0/1648918633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6298108d90 0x7f629819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.509+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 >> 172.21.15.27:0/1648918633 conn(0x7f6298077d30 msgr2=0x7f629810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.510+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 shutdown_connections 2024-04-26T21:11:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.510+0000 7f629d200640 1 -- 172.21.15.27:0/1648918633 wait complete. 2024-04-26T21:11:56.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:56 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4287483786' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:11:56.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:56.531 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.751+0000 7fd613e00640 1 -- 172.21.15.27:0/378729017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101a70 msgr2=0x7fd614101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.751+0000 7fd613e00640 1 --2- 172.21.15.27:0/378729017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101a70 0x7fd614101ed0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ca0 tx=0x7fd60002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 -- 172.21.15.27:0/378729017 shutdown_connections 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 --2- 172.21.15.27:0/378729017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614102410 0x7fd614102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 --2- 172.21.15.27:0/378729017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101a70 0x7fd614101ed0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 --2- 172.21.15.27:0/378729017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614101110 0x7fd614101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 -- 172.21.15.27:0/378729017 >> 172.21.15.27:0/378729017 conn(0x7fd6140fccd0 msgr2=0x7fd6140ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 -- 172.21.15.27:0/378729017 shutdown_connections 2024-04-26T21:11:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.752+0000 7fd613e00640 1 -- 172.21.15.27:0/378729017 wait complete. 2024-04-26T21:11:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 Processor -- start 2024-04-26T21:11:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 -- start start 2024-04-26T21:11:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 0x7fd61419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614102410 0x7fd6141a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd614110430 con 0x7fd614101110 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141102b0 con 0x7fd614101a70 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.753+0000 7fd613e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141105b0 con 0x7fd614102410 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57678/0 (socket says 172.21.15.27:57678) 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 -- 172.21.15.27:0/2966236024 learned_addr learned my addr 172.21.15.27:0/2966236024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd613400640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614102410 0x7fd6141a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612000640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 0x7fd61419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 -- 172.21.15.27:0/2966236024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614102410 msgr2=0x7fd6141a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614102410 0x7fd6141a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 -- 172.21.15.27:0/2966236024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 msgr2=0x7fd61419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 0x7fd61419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612a00640 1 -- 172.21.15.27:0/2966236024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6141a27a0 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.754+0000 7fd612000640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 0x7fd61419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd612a00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd60400d950 tx=0x7fd60400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604017070 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd604009c30 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141a2a90 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60400f620 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.755+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a2fe0 con 0x7fd614101110 2024-04-26T21:11:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.756+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61410c080 con 0x7fd614101110 2024-04-26T21:11:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.759+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd604010460 con 0x7fd614101110 2024-04-26T21:11:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.760+0000 7fd60b400640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 0x7fd5f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.760+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd6040d02d0 con 0x7fd614101110 2024-04-26T21:11:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.760+0000 7fd612000640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 0x7fd5f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.761+0000 7fd612000640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 0x7fd5f80a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd61419c090 tx=0x7fd600000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.762+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd60409cae0 con 0x7fd614101110 2024-04-26T21:11:56.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.972+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd61402dad0 con 0x7fd614101110 2024-04-26T21:11:56.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.973+0000 7fd60b400640 1 -- 172.21.15.27:0/2966236024 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fd604009de0 con 0x7fd614101110 2024-04-26T21:11:56.974 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:11:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.975+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 msgr2=0x7fd5f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.975+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 0x7fd5f80a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd61419c090 tx=0x7fd600000f40 comp rx=0 tx=0).stop 2024-04-26T21:11:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.975+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 msgr2=0x7fd61419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.975+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd60400d950 tx=0x7fd60400de10 comp rx=0 tx=0).stop 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 shutdown_connections 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fd5f80a3c40 0x7fd5f80a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd614102410 0x7fd6141a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd614101a70 0x7fd61419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 --2- 172.21.15.27:0/2966236024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd614101110 0x7fd61419abd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 >> 172.21.15.27:0/2966236024 conn(0x7fd6140fccd0 msgr2=0x7fd6140fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 shutdown_connections 2024-04-26T21:11:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:56.976+0000 7fd613e00640 1 -- 172.21.15.27:0/2966236024 wait complete. 2024-04-26T21:11:56.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 -- 172.21.15.27:0/2875062023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 msgr2=0x7f8990078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 --2- 172.21.15.27:0/2875062023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f8990078990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8978009900 tx=0x7f897802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 -- 172.21.15.27:0/2875062023 shutdown_connections 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 --2- 172.21.15.27:0/2875062023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 0x7f8990079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 --2- 172.21.15.27:0/2875062023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f8990078990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 --2- 172.21.15.27:0/2875062023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 0x7f8990077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.221+0000 7f8995600640 1 -- 172.21.15.27:0/2875062023 >> 172.21.15.27:0/2875062023 conn(0x7f89900fffe0 msgr2=0x7f8990102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.222+0000 7f8995600640 1 -- 172.21.15.27:0/2875062023 shutdown_connections 2024-04-26T21:11:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.222+0000 7f8995600640 1 -- 172.21.15.27:0/2875062023 wait complete. 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 Processor -- start 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 -- start start 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 0x7f899019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 0x7f89901a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8990114860 con 0x7f8990078530 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89901146e0 con 0x7f89900772f0 2024-04-26T21:11:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8995600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89901149e0 con 0x7f8990078ed0 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8987e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8987e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57702/0 (socket says 172.21.15.27:57702) 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.223+0000 7f8987e00640 1 -- 172.21.15.27:0/4193612653 learned_addr learned my addr 172.21.15.27:0/4193612653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f898fe00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 0x7f899019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8994c00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 0x7f89901a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 -- 172.21.15.27:0/4193612653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 msgr2=0x7f89901a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 0x7f89901a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 -- 172.21.15.27:0/4193612653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 msgr2=0x7f899019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 0x7f899019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 -- 172.21.15.27:0/4193612653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a6bd0 con 0x7f8990078530 2024-04-26T21:11:57.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.224+0000 7f8987e00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8978009230 tx=0x7f89780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.225+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897803f030 con 0x7f8990078530 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.225+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8978003c90 con 0x7f8990078530 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.225+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978035070 con 0x7f8990078530 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.225+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a6e60 con 0x7f8990078530 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.225+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a72c0 con 0x7f8990078530 2024-04-26T21:11:57.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.226+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8990111680 con 0x7f8990078530 2024-04-26T21:11:57.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.229+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8978042030 con 0x7f8990078530 2024-04-26T21:11:57.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.231+0000 7f898d600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 0x7f89740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.231+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f89780f0cc0 con 0x7f8990078530 2024-04-26T21:11:57.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.231+0000 7f898fe00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 0x7f89740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.232+0000 7f898fe00640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 0x7f89740a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f898000bbe0 tx=0x7f898000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.235+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f89780f7080 con 0x7f8990078530 2024-04-26T21:11:57.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.441+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f899002dd20 con 0x7f8990078530 2024-04-26T21:11:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.442+0000 7f898d600640 1 -- 172.21.15.27:0/4193612653 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f89780bce80 con 0x7f8990078530 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 msgr2=0x7f89740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 0x7f89740a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f898000bbe0 tx=0x7f898000b390 comp rx=0 tx=0).stop 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 msgr2=0x7f899019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8978009230 tx=0x7f89780042f0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 shutdown_connections 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f89740a3c40 0x7f89740a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8990078ed0 0x7f89901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8990078530 0x7f899019f4f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 --2- 172.21.15.27:0/4193612653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89900772f0 0x7f899019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 >> 172.21.15.27:0/4193612653 conn(0x7f89900fffe0 msgr2=0x7f8990101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 shutdown_connections 2024-04-26T21:11:57.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.445+0000 7f8995600640 1 -- 172.21.15.27:0/4193612653 wait complete. 2024-04-26T21:11:57.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:57 smithi027 ceph-mon[28085]: pgmap v437: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:57.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:57 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1648918633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:57.464 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:57 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2966236024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:57.468 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:11:57.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:57 smithi204 ceph-mon[32145]: pgmap v437: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:57.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:57 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1648918633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:57.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:57 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2966236024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:57.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:57 smithi094 ceph-mon[31273]: pgmap v437: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:11:57.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:57 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1648918633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:57.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:57 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2966236024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 -- 172.21.15.27:0/4269763237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630101920 msgr2=0x7f5630101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 --2- 172.21.15.27:0/4269763237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630101920 0x7f5630101d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5618009900 tx=0x7f561802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 -- 172.21.15.27:0/4269763237 shutdown_connections 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 --2- 172.21.15.27:0/4269763237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56301022c0 0x7f5630102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 --2- 172.21.15.27:0/4269763237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630101920 0x7f5630101d80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 --2- 172.21.15.27:0/4269763237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5630100fc0 0x7f56301013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.674+0000 7f5636000640 1 -- 172.21.15.27:0/4269763237 >> 172.21.15.27:0/4269763237 conn(0x7f56300fcc00 msgr2=0x7f56300ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.675+0000 7f5636000640 1 -- 172.21.15.27:0/4269763237 shutdown_connections 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.675+0000 7f5636000640 1 -- 172.21.15.27:0/4269763237 wait complete. 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.675+0000 7f5636000640 1 Processor -- start 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 -- start start 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 0x7f563019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 0x7f56301a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5630110360 con 0x7f5630100fc0 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56301101e0 con 0x7f5630101920 2024-04-26T21:11:57.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5636000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56301104e0 con 0x7f56301022c0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5634c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5634c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57722/0 (socket says 172.21.15.27:57722) 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.676+0000 7f5634c00640 1 -- 172.21.15.27:0/2315490914 learned_addr learned my addr 172.21.15.27:0/2315490914 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5627e00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 0x7f563019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5635600640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 0x7f56301a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 -- 172.21.15.27:0/2315490914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 msgr2=0x7f56301a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 0x7f56301a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 -- 172.21.15.27:0/2315490914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 msgr2=0x7f563019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 0x7f563019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 -- 172.21.15.27:0/2315490914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56301a26b0 con 0x7f5630100fc0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5635600640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 0x7f56301a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5627e00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 0x7f563019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f5634c00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f562000d770 tx=0x7f562000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.677+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620017070 con 0x7f5630100fc0 2024-04-26T21:11:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.678+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56301a29a0 con 0x7f5630100fc0 2024-04-26T21:11:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.678+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56200054b0 con 0x7f5630100fc0 2024-04-26T21:11:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.678+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620004c60 con 0x7f5630100fc0 2024-04-26T21:11:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.678+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56301a2ec0 con 0x7f5630100fc0 2024-04-26T21:11:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.679+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563010bd10 con 0x7f5630100fc0 2024-04-26T21:11:57.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.681+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5620004e00 con 0x7f5630100fc0 2024-04-26T21:11:57.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.683+0000 7f562e000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 0x7f56140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.683+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f56200d0110 con 0x7f5630100fc0 2024-04-26T21:11:57.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.683+0000 7f5627e00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 0x7f56140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.684+0000 7f5627e00640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 0x7f56140a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5618006010 tx=0x7f5618031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.687+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f562009c920 con 0x7f5630100fc0 2024-04-26T21:11:57.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.894+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f563002dd30 con 0x7f5630100fc0 2024-04-26T21:11:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.895+0000 7f562e000640 1 -- 172.21.15.27:0/2315490914 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f562009c220 con 0x7f5630100fc0 2024-04-26T21:11:57.896 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 msgr2=0x7f56140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 0x7f56140a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5618006010 tx=0x7f5618031400 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 msgr2=0x7f563019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f562000d770 tx=0x7f562000dc30 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 shutdown_connections 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f56140a3c40 0x7f56140a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56301022c0 0x7f56301a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5630101920 0x7f563019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 --2- 172.21.15.27:0/2315490914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5630100fc0 0x7f563019aa90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 >> 172.21.15.27:0/2315490914 conn(0x7f56300fcc00 msgr2=0x7f56300fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 shutdown_connections 2024-04-26T21:11:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:57.898+0000 7f5636000640 1 -- 172.21.15.27:0/2315490914 wait complete. 2024-04-26T21:11:57.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:58.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 -- 172.21.15.27:0/1521640316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c104500 msgr2=0x7f7a7c10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 --2- 172.21.15.27:0/1521640316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c104500 0x7f7a7c10ad90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009ca0 tx=0x7f7a7402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 -- 172.21.15.27:0/1521640316 shutdown_connections 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 --2- 172.21.15.27:0/1521640316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c104500 0x7f7a7c10ad90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 --2- 172.21.15.27:0/1521640316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c103b40 0x7f7a7c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 --2- 172.21.15.27:0/1521640316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c102950 0x7f7a7c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 -- 172.21.15.27:0/1521640316 >> 172.21.15.27:0/1521640316 conn(0x7f7a7c0fdee0 msgr2=0x7f7a7c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 -- 172.21.15.27:0/1521640316 shutdown_connections 2024-04-26T21:11:58.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.115+0000 7f7a83400640 1 -- 172.21.15.27:0/1521640316 wait complete. 2024-04-26T21:11:58.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.116+0000 7f7a83400640 1 Processor -- start 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.116+0000 7f7a83400640 1 -- start start 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c103b40 0x7f7a7c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c104500 0x7f7a7c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10e360 con 0x7f7a7c102950 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10e1e0 con 0x7f7a7c104500 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a83400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10e4e0 con 0x7f7a7c103b40 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a80a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a80a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57744/0 (socket says 172.21.15.27:57744) 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a80a00640 1 -- 172.21.15.27:0/1211881096 learned_addr learned my addr 172.21.15.27:0/1211881096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a81400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c104500 0x7f7a7c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.117+0000 7f7a7be00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c103b40 0x7f7a7c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 -- 172.21.15.27:0/1211881096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c103b40 msgr2=0x7f7a7c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c103b40 0x7f7a7c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 -- 172.21.15.27:0/1211881096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c104500 msgr2=0x7f7a7c1a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c104500 0x7f7a7c1a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 -- 172.21.15.27:0/1211881096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c1a0680 con 0x7f7a7c102950 2024-04-26T21:11:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a80a00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d770 tx=0x7f7a7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70017070 con 0x7f7a7c102950 2024-04-26T21:11:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a70009c30 con 0x7f7a7c102950 2024-04-26T21:11:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1a0970 con 0x7f7a7c102950 2024-04-26T21:11:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70003c50 con 0x7f7a7c102950 2024-04-26T21:11:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.118+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a0ec0 con 0x7f7a7c102950 2024-04-26T21:11:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.120+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c069c10 con 0x7f7a7c102950 2024-04-26T21:11:58.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.122+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7a70003df0 con 0x7f7a7c102950 2024-04-26T21:11:58.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.123+0000 7f7a79600640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 0x7f7a640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.124+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7a700cffd0 con 0x7f7a7c102950 2024-04-26T21:11:58.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.124+0000 7f7a7be00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 0x7f7a640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.124+0000 7f7a7be00640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 0x7f7a640a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.128+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7a7009c8e0 con 0x7f7a7c102950 2024-04-26T21:11:58.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.378+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a7c02db70 con 0x7f7a7c102950 2024-04-26T21:11:58.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.380+0000 7f7a79600640 1 -- 172.21.15.27:0/1211881096 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7f7a7c02db70 con 0x7f7a7c102950 2024-04-26T21:11:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:58.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:11:58.382 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 msgr2=0x7f7a640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 0x7f7a640a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a68005850 comp rx=0 tx=0).stop 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 msgr2=0x7f7a7c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d770 tx=0x7f7a7000dc30 comp rx=0 tx=0).stop 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 shutdown_connections 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f7a640a3c40 0x7f7a640a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7c103b40 0x7f7a7c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7c104500 0x7f7a7c1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 --2- 172.21.15.27:0/1211881096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a7c102950 0x7f7a7c198a60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 >> 172.21.15.27:0/1211881096 conn(0x7f7a7c0fdee0 msgr2=0x7f7a7c106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 shutdown_connections 2024-04-26T21:11:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.383+0000 7f7a83400640 1 -- 172.21.15.27:0/1211881096 wait complete. 2024-04-26T21:11:58.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:58.473 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:58 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4193612653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.473 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:58 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2315490914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:58 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4193612653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:58 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2315490914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:58 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4193612653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:58 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2315490914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:11:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 -- 172.21.15.27:0/629197775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1039a0 msgr2=0x7fdedc103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 --2- 172.21.15.27:0/629197775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1039a0 0x7fdedc103e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009900 tx=0x7fdec402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 -- 172.21.15.27:0/629197775 shutdown_connections 2024-04-26T21:11:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 --2- 172.21.15.27:0/629197775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc104360 0x7fdedc10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 --2- 172.21.15.27:0/629197775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1039a0 0x7fdedc103e20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 --2- 172.21.15.27:0/629197775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1027b0 0x7fdedc102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 -- 172.21.15.27:0/629197775 >> 172.21.15.27:0/629197775 conn(0x7fdedc0fdd40 msgr2=0x7fdedc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 -- 172.21.15.27:0/629197775 shutdown_connections 2024-04-26T21:11:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.612+0000 7fdee0a00640 1 -- 172.21.15.27:0/629197775 wait complete. 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.613+0000 7fdee0a00640 1 Processor -- start 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.613+0000 7fdee0a00640 1 -- start start 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1039a0 0x7fdedc074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc104360 0x7fdedc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e190 con 0x7fdedc1027b0 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e010 con 0x7fdedc1039a0 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdee0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e310 con 0x7fdedc104360 2024-04-26T21:11:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57760/0 (socket says 172.21.15.27:57760) 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 -- 172.21.15.27:0/2130437736 learned_addr learned my addr 172.21.15.27:0/2130437736 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 -- 172.21.15.27:0/2130437736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc104360 msgr2=0x7fdedc075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc104360 0x7fdedc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 -- 172.21.15.27:0/2130437736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1039a0 msgr2=0x7fdedc074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1039a0 0x7fdedc074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.614+0000 7fdedb400640 1 -- 172.21.15.27:0/2130437736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc075af0 con 0x7fdedc1027b0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fdedb400640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00eaa0 tx=0x7fdecc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc016070 con 0x7fdedc1027b0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdecc00ce10 con 0x7fdedc1027b0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1a2950 con 0x7fdedc1027b0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011470 con 0x7fdedc1027b0 2024-04-26T21:11:58.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.615+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc1a2ea0 con 0x7fdedc1027b0 2024-04-26T21:11:58.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.616+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdedc109af0 con 0x7fdedc1027b0 2024-04-26T21:11:58.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.618+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdecc019070 con 0x7fdedc1027b0 2024-04-26T21:11:58.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.620+0000 7fded8c00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 0x7fdec00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.621+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fdecc0d0780 con 0x7fdedc1027b0 2024-04-26T21:11:58.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.621+0000 7fded3e00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 0x7fdec00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.622+0000 7fded3e00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 0x7fdec00a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdec402a910 tx=0x7fdec4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.626+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdecc09cf90 con 0x7fdedc1027b0 2024-04-26T21:11:58.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.834+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdedc063530 con 0x7fdedc1027b0 2024-04-26T21:11:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.838+0000 7fded8c00640 1 -- 172.21.15.27:0/2130437736 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7fdecc085e20 con 0x7fdedc1027b0 2024-04-26T21:11:58.839 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:11:58.840 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 msgr2=0x7fdec00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 0x7fdec00a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdec402a910 tx=0x7fdec4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 msgr2=0x7fdedc0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00eaa0 tx=0x7fdecc00ef60 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 shutdown_connections 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fdec00a3cc0 0x7fdec00a6180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc104360 0x7fdedc075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1039a0 0x7fdedc074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 --2- 172.21.15.27:0/2130437736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc1027b0 0x7fdedc0766e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.840+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 >> 172.21.15.27:0/2130437736 conn(0x7fdedc0fdd40 msgr2=0x7fdedc106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.841+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 shutdown_connections 2024-04-26T21:11:58.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:58.841+0000 7fdee0a00640 1 -- 172.21.15.27:0/2130437736 wait complete. 2024-04-26T21:11:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:58.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:59.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 -- 172.21.15.27:0/131286048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80103a50 msgr2=0x7fee80103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 --2- 172.21.15.27:0/131286048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80103a50 0x7fee80103ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 -- 172.21.15.27:0/131286048 shutdown_connections 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 --2- 172.21.15.27:0/131286048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80104410 0x7fee8010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 --2- 172.21.15.27:0/131286048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80103a50 0x7fee80103ed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 --2- 172.21.15.27:0/131286048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80102860 0x7fee80102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 -- 172.21.15.27:0/131286048 >> 172.21.15.27:0/131286048 conn(0x7fee800fddd0 msgr2=0x7fee80100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 -- 172.21.15.27:0/131286048 shutdown_connections 2024-04-26T21:11:59.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.064+0000 7fee85800640 1 -- 172.21.15.27:0/131286048 wait complete. 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 Processor -- start 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 -- start start 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80102860 0x7fee8019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 0x7fee8019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e2f0 con 0x7fee80104410 2024-04-26T21:11:59.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e170 con 0x7fee80103a50 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.065+0000 7fee85800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e470 con 0x7fee80102860 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53646/0 (socket says 172.21.15.27:53646) 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 -- 172.21.15.27:0/43546862 learned_addr learned my addr 172.21.15.27:0/43546862 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee7fe00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80102860 0x7fee8019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee77e00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 0x7fee8019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 -- 172.21.15.27:0/43546862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80102860 msgr2=0x7fee8019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80102860 0x7fee8019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 -- 172.21.15.27:0/43546862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 msgr2=0x7fee8019b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 0x7fee8019b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee84e00640 1 -- 172.21.15.27:0/43546862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801a2830 con 0x7fee80104410 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.066+0000 7fee77e00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 0x7fee8019b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee84e00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b970 tx=0x7fee7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78019070 con 0x7fee80104410 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee78014440 con 0x7fee80104410 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780075d0 con 0x7fee80104410 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801a2b20 con 0x7fee80104410 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.067+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a3020 con 0x7fee80104410 2024-04-26T21:11:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.068+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee801a32b0 con 0x7fee80104410 2024-04-26T21:11:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.072+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee7801c070 con 0x7fee80104410 2024-04-26T21:11:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.073+0000 7fee7d600640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 0x7fee640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.073+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fee780cfa90 con 0x7fee80104410 2024-04-26T21:11:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.073+0000 7fee7fe00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 0x7fee640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.074+0000 7fee7fe00640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 0x7fee640a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee70005db0 tx=0x7fee70005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.078+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee7809c3b0 con 0x7fee80104410 2024-04-26T21:11:59.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.282+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee801a3520 con 0x7fee80104410 2024-04-26T21:11:59.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.283+0000 7fee7d600640 1 -- 172.21.15.27:0/43546862 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fee78086020 con 0x7fee80104410 2024-04-26T21:11:59.285 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:11:59.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 msgr2=0x7fee640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 0x7fee640a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee70005db0 tx=0x7fee70005d40 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 msgr2=0x7fee801a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b970 tx=0x7fee7800be30 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 shutdown_connections 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fee640a3cc0 0x7fee640a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee80104410 0x7fee801a2190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103a50 0x7fee8019b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 --2- 172.21.15.27:0/43546862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee80102860 0x7fee8019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 >> 172.21.15.27:0/43546862 conn(0x7fee800fddd0 msgr2=0x7fee80074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.286+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 shutdown_connections 2024-04-26T21:11:59.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.287+0000 7fee85800640 1 -- 172.21.15.27:0/43546862 wait complete. 2024-04-26T21:11:59.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:11:59.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:59 smithi027 ceph-mon[28085]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:59.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1211881096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:11:59 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2130437736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:59 smithi204 ceph-mon[32145]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1211881096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:11:59 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2130437736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.521+0000 7f4b85400640 1 -- 172.21.15.27:0/3729800219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80103a50 msgr2=0x7f4b80103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.521+0000 7f4b85400640 1 --2- 172.21.15.27:0/3729800219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80103a50 0x7f4b80103ed0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009900 tx=0x7f4b6c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 -- 172.21.15.27:0/3729800219 shutdown_connections 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 --2- 172.21.15.27:0/3729800219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80104410 0x7f4b8010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 --2- 172.21.15.27:0/3729800219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80103a50 0x7f4b80103ed0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 --2- 172.21.15.27:0/3729800219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b80102860 0x7f4b80102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 -- 172.21.15.27:0/3729800219 >> 172.21.15.27:0/3729800219 conn(0x7f4b800fddd0 msgr2=0x7f4b80100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 -- 172.21.15.27:0/3729800219 shutdown_connections 2024-04-26T21:11:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.522+0000 7f4b85400640 1 -- 172.21.15.27:0/3729800219 wait complete. 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 Processor -- start 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 -- start start 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b80102860 0x7f4b800766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 0x7f4b80074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8010e270 con 0x7f4b80104410 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8010e0f0 con 0x7f4b80103a50 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b85400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8010e3f0 con 0x7f4b80102860 2024-04-26T21:11:59.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b84a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b84a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53656/0 (socket says 172.21.15.27:53656) 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.523+0000 7f4b84a00640 1 -- 172.21.15.27:0/2433821883 learned_addr learned my addr 172.21.15.27:0/2433821883 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b7f400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 0x7f4b80074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 -- 172.21.15.27:0/2433821883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b80102860 msgr2=0x7f4b800766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b80102860 0x7f4b800766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 -- 172.21.15.27:0/2433821883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 msgr2=0x7f4b80074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 0x7f4b80074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 -- 172.21.15.27:0/2433821883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b80075af0 con 0x7f4b80104410 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b7f400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 0x7f4b80074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b84a00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800cce0 tx=0x7f4b7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800f450 con 0x7f4b80104410 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b7800fa80 con 0x7f4b80104410 2024-04-26T21:11:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78018ad0 con 0x7f4b80104410 2024-04-26T21:11:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.524+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b801a2a90 con 0x7f4b80104410 2024-04-26T21:11:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.525+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b801a2f90 con 0x7f4b80104410 2024-04-26T21:11:59.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:59 smithi094 ceph-mon[31273]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:11:59.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1211881096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:11:59 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2130437736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:11:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.526+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b80109ca0 con 0x7f4b80104410 2024-04-26T21:11:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.528+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b7800f5f0 con 0x7f4b80104410 2024-04-26T21:11:59.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.529+0000 7f4b7cc00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 0x7f4b640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.529+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4b780d0ec0 con 0x7f4b80104410 2024-04-26T21:11:59.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.530+0000 7f4b7fe00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 0x7f4b640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.530+0000 7f4b7fe00640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 0x7f4b640a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4b801038b0 tx=0x7f4b70005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.534+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4b78014030 con 0x7f4b80104410 2024-04-26T21:11:59.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.740+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b8002de90 con 0x7f4b80104410 2024-04-26T21:11:59.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.741+0000 7f4b7cc00640 1 -- 172.21.15.27:0/2433821883 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f4b7809d6d0 con 0x7f4b80104410 2024-04-26T21:11:59.742 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 msgr2=0x7f4b640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 0x7f4b640a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4b801038b0 tx=0x7f4b70005c10 comp rx=0 tx=0).stop 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 msgr2=0x7f4b80075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800cce0 tx=0x7f4b7800a620 comp rx=0 tx=0).stop 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 shutdown_connections 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f4b640a3c40 0x7f4b640a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b80104410 0x7f4b80075260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b80103a50 0x7f4b80074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 --2- 172.21.15.27:0/2433821883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b80102860 0x7f4b800766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.743+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 >> 172.21.15.27:0/2433821883 conn(0x7f4b800fddd0 msgr2=0x7f4b80106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.744+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 shutdown_connections 2024-04-26T21:11:59.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.744+0000 7f4b85400640 1 -- 172.21.15.27:0/2433821883 wait complete. 2024-04-26T21:11:59.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.959+0000 7fe309000640 1 -- 172.21.15.27:0/1651506471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304106540 msgr2=0x7fe30410cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.959+0000 7fe309000640 1 --2- 172.21.15.27:0/1651506471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304106540 0x7fe30410cdd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009ca0 tx=0x7fe2fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 -- 172.21.15.27:0/1651506471 shutdown_connections 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 --2- 172.21.15.27:0/1651506471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304105b80 0x7fe304106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 --2- 172.21.15.27:0/1651506471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304106540 0x7fe30410cdd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 --2- 172.21.15.27:0/1651506471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304104990 0x7fe304104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 -- 172.21.15.27:0/1651506471 >> 172.21.15.27:0/1651506471 conn(0x7fe3040fff00 msgr2=0x7fe304102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 -- 172.21.15.27:0/1651506471 shutdown_connections 2024-04-26T21:11:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 -- 172.21.15.27:0/1651506471 wait complete. 2024-04-26T21:11:59.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.960+0000 7fe309000640 1 Processor -- start 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 -- start start 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304105b80 0x7fe30419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304106540 0x7fe3041a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041103c0 con 0x7fe304104990 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe304110240 con 0x7fe304105b80 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe309000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe304110540 con 0x7fe304106540 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe303400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe303400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53674/0 (socket says 172.21.15.27:53674) 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.961+0000 7fe303400640 1 -- 172.21.15.27:0/1178244375 learned_addr learned my addr 172.21.15.27:0/1178244375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe302a00640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304105b80 0x7fe30419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303e00640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304106540 0x7fe3041a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 -- 172.21.15.27:0/1178244375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304106540 msgr2=0x7fe3041a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304106540 0x7fe3041a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 -- 172.21.15.27:0/1178244375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304105b80 msgr2=0x7fe30419afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304105b80 0x7fe30419afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 -- 172.21.15.27:0/1178244375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3041a26b0 con 0x7fe304104990 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe303400640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d770 tx=0x7fe2f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4017070 con 0x7fe304104990 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2f4009c30 con 0x7fe304104990 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4003c50 con 0x7fe304104990 2024-04-26T21:11:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.962+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3041a29a0 con 0x7fe304104990 2024-04-26T21:11:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.963+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3041a2f40 con 0x7fe304104990 2024-04-26T21:11:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.964+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30410bcd0 con 0x7fe304104990 2024-04-26T21:11:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.966+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe2f4003df0 con 0x7fe304104990 2024-04-26T21:11:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.967+0000 7fe2fbe00640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 0x7fe2e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.967+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe2f40d0010 con 0x7fe304104990 2024-04-26T21:11:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.967+0000 7fe302a00640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 0x7fe2e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.968+0000 7fe302a00640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 0x7fe2e80a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005fd0 tx=0x7fe2f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:11:59.971+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe2f409c880 con 0x7fe304104990 2024-04-26T21:12:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.175+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe304063560 con 0x7fe304104990 2024-04-26T21:12:00.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.176+0000 7fe2fbe00640 1 -- 172.21.15.27:0/1178244375 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fe2f4009de0 con 0x7fe304104990 2024-04-26T21:12:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:12:00.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 msgr2=0x7fe2e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 0x7fe2e80a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005fd0 tx=0x7fe2f0005850 comp rx=0 tx=0).stop 2024-04-26T21:12:00.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 msgr2=0x7fe30419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d770 tx=0x7fe2f400dc30 comp rx=0 tx=0).stop 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 shutdown_connections 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fe2e80a3cc0 0x7fe2e80a6180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe304106540 0x7fe3041a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe304105b80 0x7fe30419afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 --2- 172.21.15.27:0/1178244375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe304104990 0x7fe30419aa90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 >> 172.21.15.27:0/1178244375 conn(0x7fe3040fff00 msgr2=0x7fe304108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 shutdown_connections 2024-04-26T21:12:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.179+0000 7fe309000640 1 -- 172.21.15.27:0/1178244375 wait complete. 2024-04-26T21:12:00.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 -- 172.21.15.27:0/196615733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428109f80 msgr2=0x7fb42810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 --2- 172.21.15.27:0/196615733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428109f80 0x7fb42810a400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb414009900 tx=0x7fb41402a410 comp rx=0 tx=0).stop 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 -- 172.21.15.27:0/196615733 shutdown_connections 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 --2- 172.21.15.27:0/196615733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb42810a940 0x7fb4281111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 --2- 172.21.15.27:0/196615733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428109f80 0x7fb42810a400 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 --2- 172.21.15.27:0/196615733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb428108d90 0x7fb4281091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 -- 172.21.15.27:0/196615733 >> 172.21.15.27:0/196615733 conn(0x7fb428077d30 msgr2=0x7fb42807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 -- 172.21.15.27:0/196615733 shutdown_connections 2024-04-26T21:12:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.401+0000 7fb427200640 1 -- 172.21.15.27:0/196615733 wait complete. 2024-04-26T21:12:00.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.402+0000 7fb427200640 1 Processor -- start 2024-04-26T21:12:00.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.402+0000 7fb427200640 1 -- start start 2024-04-26T21:12:00.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 0x7fb42819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42810a940 0x7fb4281a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281147c0 con 0x7fb428108d90 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb428114640 con 0x7fb42810a940 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb427200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb428114940 con 0x7fb428109f80 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53698/0 (socket says 172.21.15.27:53698) 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 -- 172.21.15.27:0/1082441317 learned_addr learned my addr 172.21.15.27:0/1082441317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425400640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 0x7fb42819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb426800640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42810a940 0x7fb4281a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 -- 172.21.15.27:0/1082441317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 msgr2=0x7fb42819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 0x7fb42819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 -- 172.21.15.27:0/1082441317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42810a940 msgr2=0x7fb4281a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42810a940 0x7fb4281a6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425e00640 1 -- 172.21.15.27:0/1082441317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4281a6ab0 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.403+0000 7fb425400640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 0x7fb42819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb425e00640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb41800d950 tx=0x7fb41800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418017020 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb418009c30 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4281a6da0 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800f620 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.404+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4281a7210 con 0x7fb428108d90 2024-04-26T21:12:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.405+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428110180 con 0x7fb428108d90 2024-04-26T21:12:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.407+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb41801a020 con 0x7fb428108d90 2024-04-26T21:12:00.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.409+0000 7fb41ea00640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 0x7fb40c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.409+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb4180d0000 con 0x7fb428108d90 2024-04-26T21:12:00.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.409+0000 7fb425400640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 0x7fb40c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.410+0000 7fb425400640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 0x7fb40c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb41402a930 tx=0x7fb414000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.413+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb418021070 con 0x7fb428108d90 2024-04-26T21:12:00.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/43546862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2433821883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:00 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1178244375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/43546862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2433821883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:00 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1178244375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/43546862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2433821883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:00 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1178244375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:12:00.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.662+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb42802dcf0 con 0x7fb428108d90 2024-04-26T21:12:00.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.663+0000 7fb41ea00640 1 -- 172.21.15.27:0/1082441317 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11919 (secure 0 0 0) 0x7fb418009de0 con 0x7fb428108d90 2024-04-26T21:12:00.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:12:00.664 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:09:49.487318+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34430,34517,34406,14640]},"id":1}]} 2024-04-26T21:12:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.665+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 msgr2=0x7fb40c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.665+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 0x7fb40c0a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb41402a930 tx=0x7fb414000bc0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.665+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 msgr2=0x7fb42819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.665+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb41800d950 tx=0x7fb41800de10 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 shutdown_connections 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb40c0a3c40 0x7fb40c0a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb428109f80 0x7fb42819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42810a940 0x7fb4281a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 --2- 172.21.15.27:0/1082441317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb428108d90 0x7fb42819ee90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 >> 172.21.15.27:0/1082441317 conn(0x7fb428077d30 msgr2=0x7fb42810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 shutdown_connections 2024-04-26T21:12:00.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.666+0000 7fb427200640 1 -- 172.21.15.27:0/1082441317 wait complete. 2024-04-26T21:12:00.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 -- 172.21.15.27:0/345098325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00104990 msgr2=0x7fab00104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 --2- 172.21.15.27:0/345098325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00104990 0x7fab00104db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009900 tx=0x7faaf402a410 comp rx=0 tx=0).stop 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 -- 172.21.15.27:0/345098325 shutdown_connections 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 --2- 172.21.15.27:0/345098325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00106540 0x7fab0010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 --2- 172.21.15.27:0/345098325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00105b80 0x7fab00106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 --2- 172.21.15.27:0/345098325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00104990 0x7fab00104db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.902+0000 7fab06800640 1 -- 172.21.15.27:0/345098325 >> 172.21.15.27:0/345098325 conn(0x7fab000fff00 msgr2=0x7fab00102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.903+0000 7fab06800640 1 -- 172.21.15.27:0/345098325 shutdown_connections 2024-04-26T21:12:00.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.903+0000 7fab06800640 1 -- 172.21.15.27:0/345098325 wait complete. 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 Processor -- start 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 -- start start 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 0x7fab0019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00106540 0x7fab001a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab001103c0 con 0x7fab00105b80 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab00110240 con 0x7fab00106540 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab06800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fab00110540 con 0x7fab00104990 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab04a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab04a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53718/0 (socket says 172.21.15.27:53718) 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.904+0000 7fab04a00640 1 -- 172.21.15.27:0/2684662497 learned_addr learned my addr 172.21.15.27:0/2684662497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab05400640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 0x7fab0019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab05e00640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00106540 0x7fab001a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 -- 172.21.15.27:0/2684662497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 msgr2=0x7fab0019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 0x7fab0019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 -- 172.21.15.27:0/2684662497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00106540 msgr2=0x7fab001a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00106540 0x7fab001a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 -- 172.21.15.27:0/2684662497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab001a26e0 con 0x7fab00105b80 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab05400640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 0x7fab0019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.905+0000 7fab04a00640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d950 tx=0x7faaf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.906+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0017070 con 0x7fab00105b80 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.906+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf0009c30 con 0x7fab00105b80 2024-04-26T21:12:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.906+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab001a29d0 con 0x7fab00105b80 2024-04-26T21:12:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.906+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf000f620 con 0x7fab00105b80 2024-04-26T21:12:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.906+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a2f20 con 0x7fab00105b80 2024-04-26T21:12:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.907+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0010bdd0 con 0x7fab00105b80 2024-04-26T21:12:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.909+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faaf0010460 con 0x7fab00105b80 2024-04-26T21:12:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.910+0000 7faafe000640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 0x7faae80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.911+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7faaf00d02c0 con 0x7fab00105b80 2024-04-26T21:12:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.911+0000 7fab05400640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 0x7faae80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.912+0000 7fab05400640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 0x7faae80a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faaf40057c0 tx=0x7faaf4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:00.915+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faaf009cad0 con 0x7fab00105b80 2024-04-26T21:12:01.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.128+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab0002dcf0 con 0x7fab00105b80 2024-04-26T21:12:01.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.133+0000 7faafe000640 1 -- 172.21.15.27:0/2684662497 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22670 (secure 0 0 0) 0x7faaf0009de0 con 0x7fab00105b80 2024-04-26T21:12:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:12:01.134 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:10:03.670913+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.135+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 msgr2=0x7faae80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.135+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 0x7faae80a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faaf40057c0 tx=0x7faaf4031400 comp rx=0 tx=0).stop 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.135+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 msgr2=0x7fab0019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.135+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d950 tx=0x7faaf000de10 comp rx=0 tx=0).stop 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 shutdown_connections 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7faae80a3c40 0x7faae80a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab00106540 0x7fab001a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00105b80 0x7fab0019b000 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 --2- 172.21.15.27:0/2684662497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab00104990 0x7fab0019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 >> 172.21.15.27:0/2684662497 conn(0x7fab000fff00 msgr2=0x7fab00108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 shutdown_connections 2024-04-26T21:12:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.136+0000 7fab06800640 1 -- 172.21.15.27:0/2684662497 wait complete. 2024-04-26T21:12:01.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:09:40.863524+0000', 'modified': '2024-04-26T21:09:49.487318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34472, 'mds_2': 34430, 'mds_3': 34406, 'mds_4': 14640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6839/4118119230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 4118119230}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 4118119230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.27:6841/1078046722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 1078046722}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 1078046722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/3519578339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 3519578339}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 3519578339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1241203213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1241203213}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1241203213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/743097112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 743097112}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 743097112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6839/1122553339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1122553339}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1122553339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.94:6833/2182356287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6832', 'nonce': 2182356287}, {'type': 'v1', 'addr': '172.21.15.94:6833', 'nonce': 2182356287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.94:6835/410689838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 410689838}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 410689838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.94:6837/396354174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 396354174}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 396354174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6839/12233135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 12233135}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 12233135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34472, 34430, 34517, 34406, 14640]} 2024-04-26T21:12:01.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:12:01.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 -- 172.21.15.27:0/4216206210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9810ab60 msgr2=0x7f6d981113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 --2- 172.21.15.27:0/4216206210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9810ab60 0x7f6d981113f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009ca0 tx=0x7f6d9002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 -- 172.21.15.27:0/4216206210 shutdown_connections 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 --2- 172.21.15.27:0/4216206210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 0x7f6d9810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 --2- 172.21.15.27:0/4216206210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d9810ab60 0x7f6d981113f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 --2- 172.21.15.27:0/4216206210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d98108fb0 0x7f6d981093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 -- 172.21.15.27:0/4216206210 >> 172.21.15.27:0/4216206210 conn(0x7f6d98077f40 msgr2=0x7f6d9807a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.371+0000 7f6d9e200640 1 -- 172.21.15.27:0/4216206210 shutdown_connections 2024-04-26T21:12:01.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.372+0000 7f6d9e200640 1 -- 172.21.15.27:0/4216206210 wait complete. 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.372+0000 7f6d9e200640 1 Processor -- start 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 -- start start 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 0x7f6d9819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 0x7f6d981a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981149f0 con 0x7f6d98108fb0 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98114870 con 0x7f6d9810ab60 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d9e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98114b70 con 0x7f6d9810a1a0 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53724/0 (socket says 172.21.15.27:53724) 2024-04-26T21:12:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.373+0000 7f6d97400640 1 -- 172.21.15.27:0/1509482033 learned_addr learned my addr 172.21.15.27:0/1509482033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 0x7f6d981a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 0x7f6d9819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 -- 172.21.15.27:0/1509482033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 msgr2=0x7f6d9819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 0x7f6d9819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 -- 172.21.15.27:0/1509482033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 msgr2=0x7f6d981a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 0x7f6d981a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 -- 172.21.15.27:0/1509482033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d981a6cc0 con 0x7f6d98108fb0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 0x7f6d981a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d97400640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800d770 tx=0x7f6d8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d94c00640 1 -- 172.21.15.27:0/1509482033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88017070 con 0x7f6d98108fb0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.374+0000 7f6d94c00640 1 -- 172.21.15.27:0/1509482033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d88009c30 con 0x7f6d98108fb0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.375+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d981a6fb0 con 0x7f6d98108fb0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.375+0000 7f6d94c00640 1 -- 172.21.15.27:0/1509482033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003c50 con 0x7f6d98108fb0 2024-04-26T21:12:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.375+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981a7410 con 0x7f6d98108fb0 2024-04-26T21:12:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 Processor -- start 2024-04-26T21:12:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 -- start start 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ace510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015ad35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558015a32600 con 0x558015ad3c50 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580159cf100 con 0x558015aceae0 2024-04-26T21:12:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.379+0000 7f6da1199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558015a2e5c0 con 0x558015ace0f0 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d94c00640 1 -- 172.21.15.27:0/1509482033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6d88003df0 con 0x7f6d98108fb0 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d8fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d8fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53732/0 (socket says 172.21.15.27:53732) 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d8fe00640 1 -- 172.21.15.27:0/243632170 learned_addr learned my addr 172.21.15.27:0/243632170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d97e00640 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ace510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.380+0000 7f6d97400640 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015ad35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 -- 172.21.15.27:0/243632170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 msgr2=0x558015ace510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ace510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 -- 172.21.15.27:0/243632170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 msgr2=0x558015ad35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015ad35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 -- 172.21.15.27:0/243632170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580157442d0 con 0x558015ad3c50 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000bc90 tx=0x7f6d8000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb27f20a0a69e20e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d87e00640 1 -- 172.21.15.27:0/243632170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80015070 con 0x558015ad3c50 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d87e00640 1 -- 172.21.15.27:0/243632170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d8001a8f0 con 0x558015ad3c50 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.381+0000 7f6d87e00640 1 -- 172.21.15.27:0/243632170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d800117c0 con 0x558015ad3c50 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6d94c00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 0x7f6d7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6d94c00640 1 -- 172.21.15.27:0/1509482033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6d880cffd0 con 0x7f6d98108fb0 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 0x7f6d7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 -- 172.21.15.27:0/243632170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 msgr2=0x558015ad60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000bc90 tx=0x7f6d8000f730 comp rx=0 tx=0).stop 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 -- 172.21.15.27:0/243632170 shutdown_connections 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ad60e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015ad35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 --2- 172.21.15.27:0/243632170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ace510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.382+0000 7f6da1199740 1 -- 172.21.15.27:0/243632170 >> 172.21.15.27:0/243632170 conn(0x558015a62d10 msgr2=0x558015ac9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.383+0000 7f6da1199740 1 -- 172.21.15.27:0/243632170 shutdown_connections 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.383+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 0x7f6d7c0a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d9002ade0 tx=0x7f6d90005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.383+0000 7f6da1199740 1 -- 172.21.15.27:0/243632170 wait complete. 2024-04-26T21:12:01.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.383+0000 7f6da1199740 1 -- start start 2024-04-26T21:12:01.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.383+0000 7f6da1199740 1 Processor -- start 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ace0f0 0x558015ad8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 0x558015adfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558015ad6c00 con 0x558015aceae0 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580159b9f70 con 0x558015ace0f0 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558015ad6d80 con 0x558015ad3c50 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6da1199740 10 client.0 _ll_register_callbacks cb 0x558015a28480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6d8fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6d8fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53736/0 (socket says 172.21.15.27:53736) 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6d8fe00640 1 -- 172.21.15.27:0/1094093976 learned_addr learned my addr 172.21.15.27:0/1094093976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.384+0000 7f6d97400640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ace0f0 0x558015ad8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 0x558015adfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 -- 172.21.15.27:0/1094093976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 msgr2=0x558015adfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 0x558015adfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 -- 172.21.15.27:0/1094093976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ace0f0 msgr2=0x558015ad8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ace0f0 0x558015ad8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 -- 172.21.15.27:0/1094093976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801580e6b0 con 0x558015aceae0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d97e00640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 0x558015adfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8001a690 tx=0x7f6d8001a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a3c9a66f88d88e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80028060 con 0x558015aceae0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d8001a8f0 con 0x558015aceae0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.385+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6d8001acb0 con 0x558015aceae0 2024-04-26T21:12:01.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d80015070 con 0x558015aceae0 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6da1199740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558015ae2b30 con 0x558015aceae0 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d8001a8f0 con 0x558015aceae0 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d80004ac0 con 0x558015aceae0 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6da1199740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.386+0000 7f6da1199740 10 client.15252 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6da1199740 20 client.15252 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015ae4d40 0x558015ae61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6da1199740 4 client.15252 mds_command: new command op to 14640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 --> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558015a40210 con 0x558015ae4d40 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6d97400640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015ae4d40 0x558015ae61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6d84c00640 10 client.15252 ms_handle_connect on v2:172.21.15.27:6840/1078046722 2024-04-26T21:12:01.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.387+0000 7f6d97400640 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015ae4d40 0x558015ae61b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f9f0faba90104d0 server_cookie=2ae878ff5d1e67ab in_seq=0 out_seq=0 2024-04-26T21:12:01.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.388+0000 7f6d84c00640 1 -- 172.21.15.27:0/1094093976 <== mds.4 v2:172.21.15.27:6840/1078046722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558015a40210 con 0x558015ae4d40 2024-04-26T21:12:01.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.388+0000 7f6d84c00640 10 client.15252 handle_command_reply: tid=0 2024-04-26T21:12:01.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.388+0000 7f6da1199740 1 client.15252 shutdown 2024-04-26T21:12:01.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.388+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 msgr2=0x558015ad8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.388+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8001a690 tx=0x7f6d8001a6c0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 shutdown_connections 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ad3c50 0x558015adfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015ae4d40 0x558015ae61b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015aceae0 0x558015ad8ac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 --2- 172.21.15.27:0/1094093976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ace0f0 0x558015ad8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6d84c00640 0 client.15252 ms_handle_reset on v2:172.21.15.27:6840/1078046722 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 >> 172.21.15.27:0/1094093976 conn(0x558015a62d10 msgr2=0x558015ac8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 shutdown_connections 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 1 -- 172.21.15.27:0/1094093976 wait complete. 2024-04-26T21:12:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.389+0000 7f6da1199740 20 client.15252 trim_cache size 0 max 16384 2024-04-26T21:12:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.409+0000 7f6da1199740 1 Processor -- start 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.409+0000 7f6da1199740 1 -- start start 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ac8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015adfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558015ae44f0 con 0x558015ad3c50 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558015ae4bc0 con 0x558015aceae0 2024-04-26T21:12:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6da1199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558015ae5290 con 0x558015ace0f0 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53738/0 (socket says 172.21.15.27:53738) 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6d97e00640 1 -- 172.21.15.27:0/3727847602 learned_addr learned my addr 172.21.15.27:0/3727847602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015adfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.410+0000 7f6d97400640 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ac8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 -- 172.21.15.27:0/3727847602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 msgr2=0x558015ac8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ac8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 -- 172.21.15.27:0/3727847602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 msgr2=0x558015adfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015adfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 -- 172.21.15.27:0/3727847602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558015ad96b0 con 0x558015ad3c50 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d97e00640 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d90035000 tx=0x7f6d90040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc95f821c167a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d87400640 1 -- 172.21.15.27:0/3727847602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90004100 con 0x558015ad3c50 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d87400640 1 -- 172.21.15.27:0/3727847602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d90036050 con 0x558015ad3c50 2024-04-26T21:12:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.411+0000 7f6d87400640 1 -- 172.21.15.27:0/3727847602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90030a90 con 0x558015ad3c50 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 -- 172.21.15.27:0/3727847602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 msgr2=0x558015ac8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d90035000 tx=0x7f6d90040000 comp rx=0 tx=0).stop 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 -- 172.21.15.27:0/3727847602 shutdown_connections 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ad3c50 0x558015ac8b60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015aceae0 0x558015adfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 --2- 172.21.15.27:0/3727847602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015ace0f0 0x558015ac8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 -- 172.21.15.27:0/3727847602 >> 172.21.15.27:0/3727847602 conn(0x558015a62d10 msgr2=0x558015acff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 -- 172.21.15.27:0/3727847602 shutdown_connections 2024-04-26T21:12:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.412+0000 7f6da1199740 1 -- 172.21.15.27:0/3727847602 wait complete. 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 -- start start 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 Processor -- start 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015aceae0 0x558015adc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ad3c50 0x558015adf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558015ae6030 con 0x558015ace0f0 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558015ae5960 con 0x558015ad3c50 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558015ae6700 con 0x558015aceae0 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6da1199740 10 client.0 _ll_register_callbacks cb 0x7f6d90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.413+0000 7f6d97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53742/0 (socket says 172.21.15.27:53742) 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 -- 172.21.15.27:0/2820734455 learned_addr learned my addr 172.21.15.27:0/2820734455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97e00640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ad3c50 0x558015adf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015aceae0 0x558015adc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 -- 172.21.15.27:0/2820734455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015aceae0 msgr2=0x558015adc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015aceae0 0x558015adc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 -- 172.21.15.27:0/2820734455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ad3c50 msgr2=0x558015adf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ad3c50 0x558015adf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 -- 172.21.15.27:0/2820734455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558015ad66b0 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d97400640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d8800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7efc3d97cabdef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.414+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88020040 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6d88098850 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d90036050 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d880a0590 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558015b4b620 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d90036050 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d880a49c0 con 0x558015ace0f0 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 10 client.15258 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 20 client.15258 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015b50230 0x558015b4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 4 client.15258 mds_command: new command op to 14640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.415+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 --> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] -- command(tid 0: {"prefix": "status"}) -- 0x558015a3a080 con 0x558015b50230 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015b50230 0x558015b4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6d8fe00640 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015b50230 0x558015b4ef40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bcfce65a0afd730f server_cookie=2022547a5b277d86 in_seq=0 out_seq=0 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6d84c00640 10 client.15258 ms_handle_connect on v2:172.21.15.27:6840/1078046722 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6d84c00640 1 -- 172.21.15.27:0/2820734455 <== mds.4 v2:172.21.15.27:6840/1078046722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558015a3a080 con 0x558015b50230 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6d84c00640 10 client.15258 handle_command_reply: tid=0 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.416+0000 7f6da1199740 1 client.15258 shutdown 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 msgr2=0x558015adbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d8800df10 comp rx=0 tx=0).stop 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 shutdown_connections 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558015aceae0 0x558015adc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:6840/1078046722,v1:172.21.15.27:6841/1078046722] conn(0x558015b50230 0x558015b4ef40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558015ad3c50 0x558015adf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 --2- 172.21.15.27:0/2820734455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558015ace0f0 0x558015adbbd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 >> 172.21.15.27:0/2820734455 conn(0x558015a62d10 msgr2=0x558015ac8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6d84c00640 0 client.15258 ms_handle_reset on v2:172.21.15.27:6840/1078046722 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 shutdown_connections 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 1 -- 172.21.15.27:0/2820734455 wait complete. 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.417+0000 7f6da1199740 20 client.15258 trim_cache size 0 max 16384 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-04-26T21:12:01.422 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 14640, 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 132.93938866799999, 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 149.25921712600001 2024-04-26T21:12:01.423 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 msgr2=0x7f6d7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 0x7f6d7c0a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d9002ade0 tx=0x7f6d90005dc0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 msgr2=0x7f6d9819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800d770 tx=0x7f6d8800dc30 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 shutdown_connections 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f6d7c0a3c40 0x7f6d7c0a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d9810a1a0 0x7f6d9819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d9810ab60 0x7f6d981a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 --2- 172.21.15.27:0/1509482033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d98108fb0 0x7f6d9819f0a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 >> 172.21.15.27:0/1509482033 conn(0x7f6d98077f40 msgr2=0x7f6d9810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 shutdown_connections 2024-04-26T21:12:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.418+0000 7f6d9e200640 1 -- 172.21.15.27:0/1509482033 wait complete. 2024-04-26T21:12:01.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:01 smithi027 ceph-mon[28085]: pgmap v439: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:12:01.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:01 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1082441317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:01 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2684662497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:12:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:01 smithi204 ceph-mon[32145]: pgmap v439: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:12:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:01 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1082441317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:01 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2684662497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:01 smithi094 ceph-mon[31273]: pgmap v439: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:12:01.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:01 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1082441317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:01 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2684662497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 -- 172.21.15.27:0/567453403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105d70 msgr2=0x7fba9c1061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 --2- 172.21.15.27:0/567453403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105d70 0x7fba9c1061f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba88009900 tx=0x7fba8802a410 comp rx=0 tx=0).stop 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 -- 172.21.15.27:0/567453403 shutdown_connections 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 --2- 172.21.15.27:0/567453403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 0x7fba9c10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 --2- 172.21.15.27:0/567453403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105d70 0x7fba9c1061f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 --2- 172.21.15.27:0/567453403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c104b80 0x7fba9c104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 -- 172.21.15.27:0/567453403 >> 172.21.15.27:0/567453403 conn(0x7fba9c1000f0 msgr2=0x7fba9c102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 -- 172.21.15.27:0/567453403 shutdown_connections 2024-04-26T21:12:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.655+0000 7fbaa1600640 1 -- 172.21.15.27:0/567453403 wait complete. 2024-04-26T21:12:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.656+0000 7fbaa1600640 1 Processor -- start 2024-04-26T21:12:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.656+0000 7fbaa1600640 1 -- start start 2024-04-26T21:12:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c105d70 0x7fba9c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 0x7fba9c1a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c110560 con 0x7fba9c104b80 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1103e0 con 0x7fba9c106730 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fbaa1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1106e0 con 0x7fba9c105d70 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fba9aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fba9aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53750/0 (socket says 172.21.15.27:53750) 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fba9aa00640 1 -- 172.21.15.27:0/1821989158 learned_addr learned my addr 172.21.15.27:0/1821989158 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fba9a000640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c105d70 0x7fba9c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.657+0000 7fba9b400640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 0x7fba9c1a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 -- 172.21.15.27:0/1821989158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c105d70 msgr2=0x7fba9c19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c105d70 0x7fba9c19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 -- 172.21.15.27:0/1821989158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 msgr2=0x7fba9c1a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 0x7fba9c1a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 -- 172.21.15.27:0/1821989158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c1a2920 con 0x7fba9c104b80 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba9aa00640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d950 tx=0x7fba8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba93400640 1 -- 172.21.15.27:0/1821989158 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c017070 con 0x7fba9c104b80 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c1a2c10 con 0x7fba9c104b80 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba93400640 1 -- 172.21.15.27:0/1821989158 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8c009c30 con 0x7fba9c104b80 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c1a3110 con 0x7fba9c104b80 2024-04-26T21:12:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.658+0000 7fba93400640 1 -- 172.21.15.27:0/1821989158 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c021060 con 0x7fba9c104b80 2024-04-26T21:12:01.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 Processor -- start 2024-04-26T21:12:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fba93400640 1 -- 172.21.15.27:0/1821989158 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba8c01a070 con 0x7fba9c104b80 2024-04-26T21:12:01.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 -- start start 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x56001794e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x5600179510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600178ad600 con 0x5600179490f0 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56001784a100 con 0x560017949ae0 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.662+0000 7fbaa4398740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5600178a95c0 con 0x56001794ec50 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53762/0 (socket says 172.21.15.27:53762) 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 -- 172.21.15.27:0/560063998 learned_addr learned my addr 172.21.15.27:0/560063998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9aa00640 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x56001794e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9a000640 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x5600179510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 -- 172.21.15.27:0/560063998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 msgr2=0x5600179510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x5600179510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 -- 172.21.15.27:0/560063998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 msgr2=0x56001794e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x56001794e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 -- 172.21.15.27:0/560063998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600175bf2d0 con 0x5600179490f0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba9b400640 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ee0 tx=0x7fba94010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4656c57fbc26a41a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.663+0000 7fba93400640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 0x7fba800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.664+0000 7fba87400640 1 -- 172.21.15.27:0/560063998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94016070 con 0x5600179490f0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.664+0000 7fba87400640 1 -- 172.21.15.27:0/560063998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba94004d20 con 0x5600179490f0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.664+0000 7fba87400640 1 -- 172.21.15.27:0/560063998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940126b0 con 0x5600179490f0 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.664+0000 7fba93400640 1 -- 172.21.15.27:0/1821989158 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fba8c0d0260 con 0x7fba9c104b80 2024-04-26T21:12:01.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.664+0000 7fba9aa00640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 0x7fba800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- 172.21.15.27:0/560063998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 msgr2=0x560017949510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ee0 tx=0x7fba94010740 comp rx=0 tx=0).stop 2024-04-26T21:12:01.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fba9aa00640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 0x7fba800a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x560017949ee0 tx=0x7fba8c00bf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- 172.21.15.27:0/560063998 shutdown_connections 2024-04-26T21:12:01.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x5600179510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x56001794e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 --2- 172.21.15.27:0/560063998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5600179490f0 0x560017949510 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- 172.21.15.27:0/560063998 >> 172.21.15.27:0/560063998 conn(0x5600178ddd10 msgr2=0x560017944b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- 172.21.15.27:0/560063998 shutdown_connections 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- 172.21.15.27:0/560063998 wait complete. 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.665+0000 7fbaa4398740 1 -- start start 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 Processor -- start 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x560017953550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560017951c00 con 0x560017949ae0 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560017834f70 con 0x5600179490f0 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560017951d80 con 0x56001794ec50 2024-04-26T21:12:01.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fbaa4398740 10 client.0 _ll_register_callbacks cb 0x5600178a3480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fba9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fba9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53770/0 (socket says 172.21.15.27:53770) 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.666+0000 7fba9a000640 1 -- 172.21.15.27:0/1981998977 learned_addr learned my addr 172.21.15.27:0/1981998977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9b400640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x560017953550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9b400640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 -- 172.21.15.27:0/1981998977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 msgr2=0x56001795aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 -- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 msgr2=0x560017953550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x560017953550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 -- 172.21.15.27:0/1981998977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600176896b0 con 0x560017949ae0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9b400640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba9a000640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba88005940 tx=0x7fba88031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f4ee02ee9c244b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba7be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.667+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88035030 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba94004d20 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fba88004240 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba88004440 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fbaa4398740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56001795db70 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba94004d20 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fba8803c740 con 0x560017949ae0 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.668+0000 7fbaa4398740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fbaa4398740 10 client.15270 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fbaa4398740 20 client.15270 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179601d0 0x560017961640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fbaa4398740 4 client.15270 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:01.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 --> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600178bb210 con 0x5600179601d0 2024-04-26T21:12:01.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.669+0000 7fba9b400640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179601d0 0x560017961640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.670+0000 7fba9b400640 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179601d0 0x560017961640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0f8efec6abdd086 server_cookie=3f51558c6c4a645c in_seq=0 out_seq=0 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.670+0000 7fba7be00640 10 client.15270 ms_handle_connect on v2:172.21.15.94:6832/2182356287 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.672+0000 7fba7be00640 1 -- 172.21.15.27:0/1981998977 <== mds.3 v2:172.21.15.94:6832/2182356287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600178bb210 con 0x5600179601d0 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.672+0000 7fba7be00640 10 client.15270 handle_command_reply: tid=0 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.672+0000 7fbaa4398740 1 client.15270 shutdown 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 msgr2=0x560017953a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba88005940 tx=0x7fba88031820 comp rx=0 tx=0).stop 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 shutdown_connections 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179601d0 0x560017961640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fba7be00640 0 client.15270 ms_handle_reset on v2:172.21.15.94:6832/2182356287 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x560017953a90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 --2- 172.21.15.27:0/1981998977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x560017953550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 >> 172.21.15.27:0/1981998977 conn(0x5600178ddd10 msgr2=0x560017943eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 shutdown_connections 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 1 -- 172.21.15.27:0/1981998977 wait complete. 2024-04-26T21:12:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.673+0000 7fbaa4398740 20 client.15270 trim_cache size 0 max 16384 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.693+0000 7fbaa4398740 1 Processor -- start 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 -- start start 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x56001794aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56001795f980 con 0x560017949ae0 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560017960050 con 0x5600179490f0 2024-04-26T21:12:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fbaa4398740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560017960720 con 0x56001794ec50 2024-04-26T21:12:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fba9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fba9a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53772/0 (socket says 172.21.15.27:53772) 2024-04-26T21:12:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.694+0000 7fba9a000640 1 -- 172.21.15.27:0/2303561299 learned_addr learned my addr 172.21.15.27:0/2303561299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9b400640 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x56001794aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 -- 172.21.15.27:0/2303561299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 msgr2=0x56001795aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 -- 172.21.15.27:0/2303561299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 msgr2=0x56001794aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x56001794aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 -- 172.21.15.27:0/2303561299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600179408a0 con 0x560017949ae0 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9b400640 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x56001794aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba9a000640 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721af3934670a06d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba86a00640 1 -- 172.21.15.27:0/2303561299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88035040 con 0x560017949ae0 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba86a00640 1 -- 172.21.15.27:0/2303561299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba88053070 con 0x560017949ae0 2024-04-26T21:12:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.695+0000 7fba86a00640 1 -- 172.21.15.27:0/2303561299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88046a80 con 0x560017949ae0 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 -- 172.21.15.27:0/2303561299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 msgr2=0x56001794b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88031890 comp rx=0 tx=0).stop 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 -- 172.21.15.27:0/2303561299 shutdown_connections 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56001794ec50 0x56001795aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560017949ae0 0x56001794b3e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 --2- 172.21.15.27:0/2303561299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600179490f0 0x56001794aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 -- 172.21.15.27:0/2303561299 >> 172.21.15.27:0/2303561299 conn(0x5600178ddd10 msgr2=0x56001795efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 -- 172.21.15.27:0/2303561299 shutdown_connections 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.696+0000 7fbaa4398740 1 -- 172.21.15.27:0/2303561299 wait complete. 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 -- start start 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 Processor -- start 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5600179490f0 0x560017956ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x560017957020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5600179614c0 con 0x56001794ec50 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560017960df0 con 0x560017949ae0 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560017961b90 con 0x5600179490f0 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.697+0000 7fbaa4398740 10 client.0 _ll_register_callbacks cb 0x7fba88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53774/0 (socket says 172.21.15.27:53774) 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 -- 172.21.15.27:0/4280519759 learned_addr learned my addr 172.21.15.27:0/4280519759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5600179490f0 0x560017956ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9a000640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x560017957020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 -- 172.21.15.27:0/4280519759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5600179490f0 msgr2=0x560017956ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5600179490f0 0x560017956ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 -- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 msgr2=0x560017957020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x560017957020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 -- 172.21.15.27:0/4280519759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001795a630 con 0x56001794ec50 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9a000640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x560017957020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba9b400640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ee0 tx=0x7fba9401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad59e9931eadfa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.698+0000 7fba7be00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94016040 con 0x56001794ec50 2024-04-26T21:12:01.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba88053070 con 0x56001794ec50 2024-04-26T21:12:01.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fba940275c0 con 0x56001794ec50 2024-04-26T21:12:01.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba94039050 con 0x56001794ec50 2024-04-26T21:12:01.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fbaa4398740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600179cf1f0 con 0x56001794ec50 2024-04-26T21:12:01.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.699+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba88053070 con 0x56001794ec50 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fba9400f040 con 0x56001794ec50 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 10 client.15276 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 20 client.15276 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179d3c40 0x5600179d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 4 client.15276 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 --> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] -- command(tid 0: {"prefix": "status"}) -- 0x5600178b5080 con 0x5600179d3c40 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.700+0000 7fba9a000640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179d3c40 0x5600179d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.701+0000 7fba7be00640 10 client.15276 ms_handle_connect on v2:172.21.15.94:6832/2182356287 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.701+0000 7fba9a000640 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179d3c40 0x5600179d2950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4af335729ea85e13 server_cookie=f96af2dd53355e81 in_seq=0 out_seq=0 2024-04-26T21:12:01.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fba7be00640 1 -- 172.21.15.27:0/4280519759 <== mds.3 v2:172.21.15.94:6832/2182356287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5600178b5080 con 0x5600179d3c40 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fba7be00640 10 client.15276 handle_command_reply: tid=0 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 client.15276 shutdown 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 msgr2=0x56001795a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ee0 tx=0x7fba9401c040 comp rx=0 tx=0).stop 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 shutdown_connections 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:6832/2182356287,v1:172.21.15.94:6833/2182356287] conn(0x5600179d3c40 0x5600179d2950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56001794ec50 0x56001795a040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560017949ae0 0x560017957020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 --2- 172.21.15.27:0/4280519759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5600179490f0 0x560017956ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fba7be00640 0 client.15276 ms_handle_reset on v2:172.21.15.94:6832/2182356287 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 >> 172.21.15.27:0/4280519759 conn(0x5600178ddd10 msgr2=0x56001794aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 shutdown_connections 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 1 -- 172.21.15.27:0/4280519759 wait complete. 2024-04-26T21:12:01.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.702+0000 7fbaa4398740 20 client.15276 trim_cache size 0 max 16384 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34406, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 134.12356359099999, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:12:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 185.616058603 2024-04-26T21:12:01.708 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:01.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 msgr2=0x7fba800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 0x7fba800a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x560017949ee0 tx=0x7fba8c00bf60 comp rx=0 tx=0).stop 2024-04-26T21:12:01.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 msgr2=0x7fba9c19ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d950 tx=0x7fba8c00de10 comp rx=0 tx=0).stop 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 shutdown_connections 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fba800a3c40 0x7fba800a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba9c105d70 0x7fba9c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba9c106730 0x7fba9c1a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 --2- 172.21.15.27:0/1821989158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c104b80 0x7fba9c19ad10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 >> 172.21.15.27:0/1821989158 conn(0x7fba9c1000f0 msgr2=0x7fba9c109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 shutdown_connections 2024-04-26T21:12:01.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.703+0000 7fbaa1600640 1 -- 172.21.15.27:0/1821989158 wait complete. 2024-04-26T21:12:01.722 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:12:01.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 -- 172.21.15.27:0/2456025857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78104990 msgr2=0x7fcf78104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 --2- 172.21.15.27:0/2456025857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78104990 0x7fcf78104db0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009900 tx=0x7fcf6802a410 comp rx=0 tx=0).stop 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 -- 172.21.15.27:0/2456025857 shutdown_connections 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 --2- 172.21.15.27:0/2456025857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78106540 0x7fcf7810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 --2- 172.21.15.27:0/2456025857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78105b80 0x7fcf78106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 --2- 172.21.15.27:0/2456025857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78104990 0x7fcf78104db0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.939+0000 7fcf7d400640 1 -- 172.21.15.27:0/2456025857 >> 172.21.15.27:0/2456025857 conn(0x7fcf780fff00 msgr2=0x7fcf78102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.940+0000 7fcf7d400640 1 -- 172.21.15.27:0/2456025857 shutdown_connections 2024-04-26T21:12:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.940+0000 7fcf7d400640 1 -- 172.21.15.27:0/2456025857 wait complete. 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 Processor -- start 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 -- start start 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78104990 0x7fcf7819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 0x7fcf781a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781103c0 con 0x7fcf78105b80 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78110240 con 0x7fcf78106540 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf7d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78110540 con 0x7fcf78104990 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53788/0 (socket says 172.21.15.27:53788) 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.941+0000 7fcf77400640 1 -- 172.21.15.27:0/3620999008 learned_addr learned my addr 172.21.15.27:0/3620999008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 0x7fcf781a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78104990 0x7fcf7819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 -- 172.21.15.27:0/3620999008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78104990 msgr2=0x7fcf7819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78104990 0x7fcf7819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 -- 172.21.15.27:0/3620999008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 msgr2=0x7fcf781a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 0x7fcf781a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 -- 172.21.15.27:0/3620999008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781a2700 con 0x7fcf78105b80 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 0x7fcf781a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf77400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d950 tx=0x7fcf6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf74c00640 1 -- 172.21.15.27:0/3620999008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x7fcf78105b80 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf74c00640 1 -- 172.21.15.27:0/3620999008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf64009c30 con 0x7fcf78105b80 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.942+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781a29f0 con 0x7fcf78105b80 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.943+0000 7fcf74c00640 1 -- 172.21.15.27:0/3620999008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400f620 con 0x7fcf78105b80 2024-04-26T21:12:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.943+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781a2f90 con 0x7fcf78105b80 2024-04-26T21:12:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.946+0000 7fcf81983740 1 Processor -- start 2024-04-26T21:12:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.946+0000 7fcf81983740 1 -- start start 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.946+0000 7fcf81983740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.946+0000 7fcf81983740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf81983740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e5e20 0x55fd149e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf81983740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd14944600 con 0x55fd149e5e20 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf81983740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd148e1100 con 0x55fd149e0cb0 2024-04-26T21:12:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf81983740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fd149405c0 con 0x55fd149e02c0 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf74c00640 1 -- 172.21.15.27:0/3620999008 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf64010460 con 0x7fcf78105b80 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:45196/0 (socket says 172.21.15.27:45196) 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf7ca00640 1 -- 172.21.15.27:0/3824184679 learned_addr learned my addr 172.21.15.27:0/3824184679 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.947+0000 7fcf77400640 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e5e20 0x55fd149e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 -- 172.21.15.27:0/3824184679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 msgr2=0x55fd149e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 -- 172.21.15.27:0/3824184679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e5e20 msgr2=0x55fd149e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e5e20 0x55fd149e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 -- 172.21.15.27:0/3824184679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd146562d0 con 0x55fd149e02c0 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.948+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009ea0 tx=0x7fcf70009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed77af3ee3630e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf6d200640 1 -- 172.21.15.27:0/3824184679 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70004d20 con 0x55fd149e02c0 2024-04-26T21:12:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf6d200640 1 -- 172.21.15.27:0/3824184679 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf700054b0 con 0x55fd149e02c0 2024-04-26T21:12:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf6d200640 1 -- 172.21.15.27:0/3824184679 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70011730 con 0x55fd149e02c0 2024-04-26T21:12:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf74c00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf74c00640 1 -- 172.21.15.27:0/3620999008 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf640d01e0 con 0x7fcf78105b80 2024-04-26T21:12:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.949+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- 172.21.15.27:0/3824184679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 msgr2=0x55fd149e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009ea0 tx=0x7fcf70009f90 comp rx=0 tx=0).stop 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- 172.21.15.27:0/3824184679 shutdown_connections 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e5e20 0x55fd149e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 --2- 172.21.15.27:0/3824184679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149e06e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- 172.21.15.27:0/3824184679 >> 172.21.15.27:0/3824184679 conn(0x55fd14974d10 msgr2=0x55fd149dbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- 172.21.15.27:0/3824184679 shutdown_connections 2024-04-26T21:12:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- 172.21.15.27:0/3824184679 wait complete. 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf680057c0 tx=0x7fcf68031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.950+0000 7fcf81983740 1 -- start start 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 Processor -- start 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd149e8dd0 con 0x55fd149e0cb0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd148cbf70 con 0x55fd149e5e20 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fd149e8f50 con 0x55fd149e02c0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.951+0000 7fcf81983740 10 client.0 _ll_register_callbacks cb 0x55fd1493a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53810/0 (socket says 172.21.15.27:53810) 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 -- 172.21.15.27:0/900696652 learned_addr learned my addr 172.21.15.27:0/900696652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77e00640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 -- 172.21.15.27:0/900696652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 msgr2=0x55fd149ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 -- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 msgr2=0x55fd149f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 -- 172.21.15.27:0/900696652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd147206b0 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf77400640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098d20 tx=0x7fcf64010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9253fcd7477134 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.952+0000 7fcf61600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64020070 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf6409b7a0 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf700054b0 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf6409bcb0 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf81983740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd149f4d50 con 0x55fd149e0cb0 2024-04-26T21:12:01.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf700054b0 con 0x55fd149e0cb0 2024-04-26T21:12:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.953+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcf64010750 con 0x55fd149e0cb0 2024-04-26T21:12:01.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 10 client.15288 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 20 client.15288 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd149f73b0 0x55fd149f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 4 client.15288 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 --> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd14952210 con 0x55fd149f73b0 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.954+0000 7fcf77e00640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd149f73b0 0x55fd149f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.955+0000 7fcf77e00640 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd149f73b0 0x55fd149f8820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a267cd441012f67 server_cookie=8950c84c87db6402 in_seq=0 out_seq=0 2024-04-26T21:12:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.955+0000 7fcf61600640 10 client.15288 ms_handle_connect on v2:172.21.15.94:6834/410689838 2024-04-26T21:12:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.957+0000 7fcf61600640 1 -- 172.21.15.27:0/900696652 <== mds.2 v2:172.21.15.94:6834/410689838 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd14952210 con 0x55fd149f73b0 2024-04-26T21:12:01.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.957+0000 7fcf61600640 10 client.15288 handle_command_reply: tid=0 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 client.15288 shutdown 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 msgr2=0x55fd149eacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf64098d20 tx=0x7fcf64010f70 comp rx=0 tx=0).stop 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 shutdown_connections 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd149f73b0 0x55fd149f8820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e0cb0 0x55fd149eacc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 --2- 172.21.15.27:0/900696652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e02c0 0x55fd149ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf61600640 0 client.15288 ms_handle_reset on v2:172.21.15.94:6834/410689838 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 >> 172.21.15.27:0/900696652 conn(0x55fd14974d10 msgr2=0x55fd149daf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 shutdown_connections 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 1 -- 172.21.15.27:0/900696652 wait complete. 2024-04-26T21:12:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.958+0000 7fcf81983740 20 client.15288 trim_cache size 0 max 16384 2024-04-26T21:12:01.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.978+0000 7fcf81983740 1 Processor -- start 2024-04-26T21:12:01.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.978+0000 7fcf81983740 1 -- start start 2024-04-26T21:12:01.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e0cb0 0x55fd149f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f6b60 con 0x55fd149e02c0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f7230 con 0x55fd149e5e20 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf81983740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f7900 con 0x55fd149e0cb0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53822/0 (socket says 172.21.15.27:53822) 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 -- 172.21.15.27:0/384061128 learned_addr learned my addr 172.21.15.27:0/384061128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77400640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e0cb0 0x55fd149f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 -- 172.21.15.27:0/384061128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e0cb0 msgr2=0x55fd149f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e0cb0 0x55fd149f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 -- 172.21.15.27:0/384061128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 msgr2=0x55fd149dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf77e00640 1 -- 172.21.15.27:0/384061128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd149e8670 con 0x55fd149e02c0 2024-04-26T21:12:01.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.979+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.980+0000 7fcf77e00640 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf680044e0 tx=0x7fcf6803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7316ba3365dbebaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.980+0000 7fcf63e00640 1 -- 172.21.15.27:0/384061128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68035040 con 0x55fd149e02c0 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.980+0000 7fcf63e00640 1 -- 172.21.15.27:0/384061128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf680527a0 con 0x55fd149e02c0 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.980+0000 7fcf63e00640 1 -- 172.21.15.27:0/384061128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003ab0 con 0x55fd149e02c0 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- 172.21.15.27:0/384061128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 msgr2=0x55fd149d6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf680044e0 tx=0x7fcf6803e000 comp rx=0 tx=0).stop 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- 172.21.15.27:0/384061128 shutdown_connections 2024-04-26T21:12:01.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e0cb0 0x55fd149f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e5e20 0x55fd149dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 --2- 172.21.15.27:0/384061128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149d6df0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- 172.21.15.27:0/384061128 >> 172.21.15.27:0/384061128 conn(0x55fd14974d10 msgr2=0x55fd149e2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- 172.21.15.27:0/384061128 shutdown_connections 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- 172.21.15.27:0/384061128 wait complete. 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.981+0000 7fcf81983740 1 -- start start 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 Processor -- start 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e5e20 0x55fd149f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f86a0 con 0x55fd149e02c0 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f7fd0 con 0x55fd149e0cb0 2024-04-26T21:12:01.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fd149f8d70 con 0x55fd149e5e20 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf81983740 10 client.0 _ll_register_callbacks cb 0x7fcf6400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53824/0 (socket says 172.21.15.27:53824) 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.982+0000 7fcf77e00640 1 -- 172.21.15.27:0/3932892891 learned_addr learned my addr 172.21.15.27:0/3932892891 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77400640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf7ca00640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e5e20 0x55fd149f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 -- 172.21.15.27:0/3932892891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e5e20 msgr2=0x55fd149f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e5e20 0x55fd149f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 -- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 msgr2=0x55fd149edb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149edb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 -- 172.21.15.27:0/3932892891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd149f10e0 con 0x55fd149e02c0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf77e00640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006010 tx=0x7fcf68005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ca98ac3ab3e07c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf61600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68035050 con 0x55fd149e02c0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.983+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf68051040 con 0x55fd149e02c0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf680527a0 con 0x55fd149e02c0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf68052ae0 con 0x55fd149e02c0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd14a5d3c0 con 0x55fd149e02c0 2024-04-26T21:12:01.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf680527a0 con 0x55fd149e02c0 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcf6803a040 con 0x55fd149e02c0 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 20 client.15294 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd14a61b00 0x55fd14a60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 4 client.15294 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.984+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 --> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] -- command(tid 0: {"prefix": "status"}) -- 0x55fd1494c080 con 0x55fd14a61b00 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.985+0000 7fcf77400640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd14a61b00 0x55fd14a60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.986+0000 7fcf77400640 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd14a61b00 0x55fd14a60810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b83a6d5ea6de04a0 server_cookie=45012cf60c5a2e7d in_seq=0 out_seq=0 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.986+0000 7fcf61600640 10 client.15294 ms_handle_connect on v2:172.21.15.94:6834/410689838 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf61600640 1 -- 172.21.15.27:0/3932892891 <== mds.2 v2:172.21.15.94:6834/410689838 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fd1494c080 con 0x55fd14a61b00 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf61600640 10 client.15294 handle_command_reply: tid=0 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 client.15294 shutdown 2024-04-26T21:12:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 msgr2=0x55fd149ed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006010 tx=0x7fcf68005a20 comp rx=0 tx=0).stop 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 shutdown_connections 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:6834/410689838,v1:172.21.15.94:6835/410689838] conn(0x55fd14a61b00 0x55fd14a60810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd149e5e20 0x55fd149f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd149e0cb0 0x55fd149edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 --2- 172.21.15.27:0/3932892891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fd149e02c0 0x55fd149ed650 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf61600640 0 client.15294 ms_handle_reset on v2:172.21.15.94:6834/410689838 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 >> 172.21.15.27:0/3932892891 conn(0x55fd14974d10 msgr2=0x55fd149da6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 shutdown_connections 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 1 -- 172.21.15.27:0/3932892891 wait complete. 2024-04-26T21:12:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.987+0000 7fcf81983740 20 client.15294 trim_cache size 0 max 16384 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34430, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 135.42469501400001, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 173.465326946 2024-04-26T21:12:01.994 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 msgr2=0x7fcf5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf680057c0 tx=0x7fcf68031400 comp rx=0 tx=0).stop 2024-04-26T21:12:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 msgr2=0x7fcf7819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d950 tx=0x7fcf6400de10 comp rx=0 tx=0).stop 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 shutdown_connections 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78106540 0x7fcf781a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78105b80 0x7fcf7819b020 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 --2- 172.21.15.27:0/3620999008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf78104990 0x7fcf7819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 >> 172.21.15.27:0/3620999008 conn(0x7fcf780fff00 msgr2=0x7fcf78108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 shutdown_connections 2024-04-26T21:12:01.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:01.988+0000 7fcf7d400640 1 -- 172.21.15.27:0/3620999008 wait complete. 2024-04-26T21:12:02.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:12:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 -- 172.21.15.27:0/1671914297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 msgr2=0x7f1b34104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 --2- 172.21.15.27:0/1671914297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b34104da0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009900 tx=0x7f1b2402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 -- 172.21.15.27:0/1671914297 shutdown_connections 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 --2- 172.21.15.27:0/1671914297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 0x7f1b34105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 --2- 172.21.15.27:0/1671914297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b3410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 --2- 172.21.15.27:0/1671914297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b34104da0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 -- 172.21.15.27:0/1671914297 >> 172.21.15.27:0/1671914297 conn(0x7f1b340fff10 msgr2=0x7f1b34102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 -- 172.21.15.27:0/1671914297 shutdown_connections 2024-04-26T21:12:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.220+0000 7f1b33200640 1 -- 172.21.15.27:0/1671914297 wait complete. 2024-04-26T21:12:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.221+0000 7f1b33200640 1 Processor -- start 2024-04-26T21:12:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.221+0000 7f1b33200640 1 -- start start 2024-04-26T21:12:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 0x7f1b3419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b341a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110380 con 0x7f1b34104980 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110200 con 0x7f1b34106530 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b33200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110500 con 0x7f1b34105b70 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53840/0 (socket says 172.21.15.27:53840) 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 -- 172.21.15.27:0/3136084906 learned_addr learned my addr 172.21.15.27:0/3136084906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31400640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 0x7f1b3419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b32800640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b341a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 -- 172.21.15.27:0/3136084906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 msgr2=0x7f1b3419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 0x7f1b3419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 -- 172.21.15.27:0/3136084906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 msgr2=0x7f1b341a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b341a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b31e00640 1 -- 172.21.15.27:0/3136084906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b341a2640 con 0x7f1b34104980 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.222+0000 7f1b32800640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b341a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b31e00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1b24031460 tx=0x7f1b24003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b2aa00640 1 -- 172.21.15.27:0/3136084906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2403f070 con 0x7f1b34104980 2024-04-26T21:12:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b2aa00640 1 -- 172.21.15.27:0/3136084906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b24031da0 con 0x7f1b34104980 2024-04-26T21:12:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b2aa00640 1 -- 172.21.15.27:0/3136084906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24035030 con 0x7f1b34104980 2024-04-26T21:12:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b341a28d0 con 0x7f1b34104980 2024-04-26T21:12:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.223+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a2dc0 con 0x7f1b34104980 2024-04-26T21:12:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.226+0000 7f1b3bd6c740 1 Processor -- start 2024-04-26T21:12:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.226+0000 7f1b2aa00640 1 -- 172.21.15.27:0/3136084906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b24042070 con 0x7f1b34104980 2024-04-26T21:12:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 -- start start 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x5570356307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557035630e20 0x5570356332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55703558f600 con 0x55703562b2c0 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55703552c100 con 0x55703562bcb0 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55703558b5c0 con 0x557035630e20 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.227+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53846/0 (socket says 172.21.15.27:53846) 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 -- 172.21.15.27:0/1836681594 learned_addr learned my addr 172.21.15.27:0/1836681594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b31e00640 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x5570356307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b31400640 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557035630e20 0x5570356332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 -- 172.21.15.27:0/1836681594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557035630e20 msgr2=0x5570356332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557035630e20 0x5570356332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 -- 172.21.15.27:0/1836681594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 msgr2=0x5570356307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x5570356307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 -- 172.21.15.27:0/1836681594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570352a12d0 con 0x55703562b2c0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b2aa00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 0x7f1b180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.228+0000 7f1b32800640 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c008e60 tx=0x7f1b2c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93cdc5a905cbc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.229+0000 7f1b31e00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 0x7f1b180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.229+0000 7f1b2aa00640 1 -- 172.21.15.27:0/3136084906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1b240f0ff0 con 0x7f1b34104980 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.229+0000 7f1b1f400640 1 -- 172.21.15.27:0/1836681594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00f830 con 0x55703562b2c0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.229+0000 7f1b1f400640 1 -- 172.21.15.27:0/1836681594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b2c018480 con 0x55703562b2c0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.229+0000 7f1b1f400640 1 -- 172.21.15.27:0/1836681594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00fe60 con 0x55703562b2c0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1836681594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 msgr2=0x55703562b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b31e00640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 0x7f1b180a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55703562c720 tx=0x7f1b24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c008e60 tx=0x7f1b2c00d710 comp rx=0 tx=0).stop 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1836681594 shutdown_connections 2024-04-26T21:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557035630e20 0x5570356332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x5570356307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1836681594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x55703562b6e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1836681594 >> 172.21.15.27:0/1836681594 conn(0x5570355bfd10 msgr2=0x557035626d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1836681594 shutdown_connections 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1836681594 wait complete. 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.230+0000 7f1b3bd6c740 1 -- start start 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 Processor -- start 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035635720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035635c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557035633dd0 con 0x557035630e20 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557035516f70 con 0x55703562bcb0 2024-04-26T21:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557035633f50 con 0x55703562b2c0 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b3bd6c740 10 client.0 _ll_register_callbacks cb 0x557035585480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53852/0 (socket says 172.21.15.27:53852) 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.231+0000 7f1b32800640 1 -- 172.21.15.27:0/1665641856 learned_addr learned my addr 172.21.15.27:0/1665641856 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b31400640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035635c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035635720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 -- 172.21.15.27:0/1665641856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 msgr2=0x557035635720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035635720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 -- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 msgr2=0x557035635c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035635c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 -- 172.21.15.27:0/1665641856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703536b6b0 con 0x557035630e20 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b32800640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e3b0 tx=0x7f1b2c018ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707a0323f2e21006 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b13e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.232+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e6b0 con 0x557035630e20 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1b2c00e850 con 0x557035630e20 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c018480 con 0x557035630e20 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c00ea50 con 0x557035630e20 2024-04-26T21:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b3bd6c740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55703563fc80 con 0x557035630e20 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b2c018480 con 0x557035630e20 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b2c023030 con 0x557035630e20 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.233+0000 7f1b3bd6c740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b3bd6c740 10 client.15306 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b3bd6c740 20 client.15306 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x557035642080 0x5570356434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b3bd6c740 4 client.15306 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 --> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55703559d210 con 0x557035642080 2024-04-26T21:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.234+0000 7f1b31400640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x557035642080 0x5570356434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.235+0000 7f1b31400640 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x557035642080 0x5570356434f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58f45288bcdd5872 server_cookie=25a0aefa19fd2c29 in_seq=0 out_seq=0 2024-04-26T21:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.235+0000 7f1b13e00640 10 client.15306 ms_handle_connect on v2:172.21.15.94:6836/396354174 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.237+0000 7f1b13e00640 1 -- 172.21.15.27:0/1665641856 <== mds.1 v2:172.21.15.94:6836/396354174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55703559d210 con 0x557035642080 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.237+0000 7f1b13e00640 10 client.15306 handle_command_reply: tid=0 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 client.15306 shutdown 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 msgr2=0x55703563cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e3b0 tx=0x7f1b2c018ba0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 shutdown_connections 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x557035642080 0x5570356434f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563cca0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035635c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/1665641856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035635720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 >> 172.21.15.27:0/1665641856 conn(0x5570355bfd10 msgr2=0x557035625f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b13e00640 0 client.15306 ms_handle_reset on v2:172.21.15.94:6836/396354174 2024-04-26T21:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 shutdown_connections 2024-04-26T21:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/1665641856 wait complete. 2024-04-26T21:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.238+0000 7f1b3bd6c740 20 client.15306 trim_cache size 0 max 16384 2024-04-26T21:12:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.258+0000 7f1b3bd6c740 1 Processor -- start 2024-04-26T21:12:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.258+0000 7f1b3bd6c740 1 -- start start 2024-04-26T21:12:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562bcb0 0x55703563cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557035630e20 0x557035635870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557035641830 con 0x55703562b2c0 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557035641f00 con 0x557035630e20 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570356425d0 con 0x55703562bcb0 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53864/0 (socket says 172.21.15.27:53864) 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 -- 172.21.15.27:0/4145359214 learned_addr learned my addr 172.21.15.27:0/4145359214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 -- 172.21.15.27:0/4145359214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562bcb0 msgr2=0x55703563cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562bcb0 0x55703563cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 -- 172.21.15.27:0/4145359214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557035630e20 msgr2=0x557035635870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557035630e20 0x557035635870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 -- 172.21.15.27:0/4145359214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703562d940 con 0x55703562b2c0 2024-04-26T21:12:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.259+0000 7f1b32800640 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c008e60 tx=0x7f1b2c008290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788f5b5035664f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b1ea00640 1 -- 172.21.15.27:0/4145359214 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c036070 con 0x55703562b2c0 2024-04-26T21:12:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b1ea00640 1 -- 172.21.15.27:0/4145359214 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b2c018480 con 0x55703562b2c0 2024-04-26T21:12:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b1ea00640 1 -- 172.21.15.27:0/4145359214 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00ebf0 con 0x55703562b2c0 2024-04-26T21:12:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/4145359214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 msgr2=0x557035635330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c008e60 tx=0x7f1b2c008290 comp rx=0 tx=0).stop 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/4145359214 shutdown_connections 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562bcb0 0x55703563cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.260+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557035630e20 0x557035635870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/4145359214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55703562b2c0 0x557035635330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/4145359214 >> 172.21.15.27:0/4145359214 conn(0x5570355bfd10 msgr2=0x557035621df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/4145359214 shutdown_connections 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/4145359214 wait complete. 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 -- start start 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.261+0000 7f1b3bd6c740 1 Processor -- start 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035638610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035638b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557035643370 con 0x557035630e20 2024-04-26T21:12:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557035642ca0 con 0x55703562bcb0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557035643a40 con 0x55703562b2c0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b3bd6c740 10 client.0 _ll_register_callbacks cb 0x7f1b2c008cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53866/0 (socket says 172.21.15.27:53866) 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 -- 172.21.15.27:0/2618603847 learned_addr learned my addr 172.21.15.27:0/2618603847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b31400640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035638b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035638610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 -- 172.21.15.27:0/2618603847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 msgr2=0x557035638610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035638610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 -- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 msgr2=0x557035638b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.262+0000 7f1b32800640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035638b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b32800640 1 -- 172.21.15.27:0/2618603847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703563c160 con 0x557035630e20 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b32800640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d750 tx=0x7f1b2c010fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f52269e9782e8be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b13e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c036070 con 0x557035630e20 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c018480 con 0x557035630e20 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1b2c026620 con 0x557035630e20 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c0268c0 con 0x557035630e20 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b3bd6c740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.263+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570356a8210 con 0x557035630e20 2024-04-26T21:12:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b2c018480 con 0x557035630e20 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b2c023040 con 0x557035630e20 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 10 client.15312 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 20 client.15312 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x5570356acbf0 0x5570356a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 4 client.15312 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.264+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 --> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] -- command(tid 0: {"prefix": "status"}) -- 0x557035597080 con 0x5570356acbf0 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.265+0000 7f1b31400640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x5570356acbf0 0x5570356a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.266+0000 7f1b31400640 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x5570356acbf0 0x5570356a9700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73233f2551822fb0 server_cookie=6ca0788604112067 in_seq=0 out_seq=0 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.266+0000 7f1b13e00640 10 client.15312 ms_handle_connect on v2:172.21.15.94:6836/396354174 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.266+0000 7f1b13e00640 1 -- 172.21.15.27:0/2618603847 <== mds.1 v2:172.21.15.94:6836/396354174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557035597080 con 0x5570356acbf0 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.266+0000 7f1b13e00640 10 client.15312 handle_command_reply: tid=0 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 client.15312 shutdown 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 msgr2=0x55703563bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d750 tx=0x7f1b2c010fe0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 shutdown_connections 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:6836/396354174,v1:172.21.15.94:6837/396354174] conn(0x5570356acbf0 0x5570356a9700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557035630e20 0x55703563bb70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703562bcb0 0x557035638b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 --2- 172.21.15.27:0/2618603847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55703562b2c0 0x557035638610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 >> 172.21.15.27:0/2618603847 conn(0x5570355bfd10 msgr2=0x557035635330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b13e00640 0 client.15312 ms_handle_reset on v2:172.21.15.94:6836/396354174 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 shutdown_connections 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 1 -- 172.21.15.27:0/2618603847 wait complete. 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.267+0000 7f1b3bd6c740 20 client.15312 trim_cache size 0 max 16384 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34472, 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:12:02.271 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 136.718641687, 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 162.207531429 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 msgr2=0x7f1b180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 0x7f1b180a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55703562c720 tx=0x7f1b24000bc0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 msgr2=0x7f1b3419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1b24031460 tx=0x7f1b24003d50 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 shutdown_connections 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f1b180a3c40 0x7f1b180a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b34105b70 0x7f1b3419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b34106530 0x7f1b341a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 --2- 172.21.15.27:0/3136084906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b34104980 0x7f1b3419aa20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 >> 172.21.15.27:0/3136084906 conn(0x7f1b340fff10 msgr2=0x7f1b34108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 shutdown_connections 2024-04-26T21:12:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.268+0000 7f1b33200640 1 -- 172.21.15.27:0/3136084906 wait complete. 2024-04-26T21:12:02.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 -- 172.21.15.27:0/386145355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 msgr2=0x7fcc941111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 --2- 172.21.15.27:0/386145355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941111d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009ca0 tx=0x7fcc8c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 -- 172.21.15.27:0/386145355 shutdown_connections 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 --2- 172.21.15.27:0/386145355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94109f80 0x7fcc9410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 --2- 172.21.15.27:0/386145355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941111d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 --2- 172.21.15.27:0/386145355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94108d90 0x7fcc941091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.491+0000 7fcc93200640 1 -- 172.21.15.27:0/386145355 >> 172.21.15.27:0/386145355 conn(0x7fcc94077d30 msgr2=0x7fcc9407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.492+0000 7fcc93200640 1 -- 172.21.15.27:0/386145355 shutdown_connections 2024-04-26T21:12:02.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.492+0000 7fcc93200640 1 -- 172.21.15.27:0/386145355 wait complete. 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 Processor -- start 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 -- start start 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94108d90 0x7fcc9419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 0x7fcc9419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941147c0 con 0x7fcc9410a940 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94114640 con 0x7fcc94109f80 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc93200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94114940 con 0x7fcc94108d90 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc92800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc92800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53894/0 (socket says 172.21.15.27:53894) 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.493+0000 7fcc92800640 1 -- 172.21.15.27:0/1492971424 learned_addr learned my addr 172.21.15.27:0/1492971424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc91400640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 0x7fcc9419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc91e00640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94108d90 0x7fcc9419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 -- 172.21.15.27:0/1492971424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94108d90 msgr2=0x7fcc9419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94108d90 0x7fcc9419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 -- 172.21.15.27:0/1492971424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 msgr2=0x7fcc9419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 0x7fcc9419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 -- 172.21.15.27:0/1492971424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc941a6a60 con 0x7fcc9410a940 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc91400640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 0x7fcc9419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.494+0000 7fcc92800640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c031040 tx=0x7fcc8c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.495+0000 7fcc8aa00640 1 -- 172.21.15.27:0/1492971424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c004040 con 0x7fcc9410a940 2024-04-26T21:12:02.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.495+0000 7fcc8aa00640 1 -- 172.21.15.27:0/1492971424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c004670 con 0x7fcc9410a940 2024-04-26T21:12:02.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.495+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc941a6cf0 con 0x7fcc9410a940 2024-04-26T21:12:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.495+0000 7fcc8aa00640 1 -- 172.21.15.27:0/1492971424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c030d60 con 0x7fcc9410a940 2024-04-26T21:12:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.495+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941a71e0 con 0x7fcc9410a940 2024-04-26T21:12:02.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.498+0000 7fcc8aa00640 1 -- 172.21.15.27:0/1492971424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc8c0305c0 con 0x7fcc9410a940 2024-04-26T21:12:02.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.498+0000 7fcc9b969740 1 Processor -- start 2024-04-26T21:12:02.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.498+0000 7fcc9b969740 1 -- start start 2024-04-26T21:12:02.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e1fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e2062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e162600 con 0x55fc2e1fecb0 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e0ff100 con 0x55fc2e203e20 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc9b969740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e15e5c0 con 0x55fc2e1fe2c0 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc91e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53900/0 (socket says 172.21.15.27:53900) 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc91e00640 1 -- 172.21.15.27:0/1880678967 learned_addr learned my addr 172.21.15.27:0/1880678967 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.499+0000 7fcc91400640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e1fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91400640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e2062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 -- 172.21.15.27:0/1880678967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 msgr2=0x55fc2e1fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e1fe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 -- 172.21.15.27:0/1880678967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 msgr2=0x55fc2e2062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e2062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 -- 172.21.15.27:0/1880678967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2de742d0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91400640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e2062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc8aa00640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 0x7fcc780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91e00640 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400bc90 tx=0x7fcc8400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30455f61f3dd6d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc8aa00640 1 -- 172.21.15.27:0/1492971424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcc8c0f1b00 con 0x7fcc9410a940 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.500+0000 7fcc91400640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 0x7fcc780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc7f000640 1 -- 172.21.15.27:0/1880678967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84015070 con 0x55fc2e1fecb0 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc7f000640 1 -- 172.21.15.27:0/1880678967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc840035f0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc7f000640 1 -- 172.21.15.27:0/1880678967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840116a0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc9b969740 1 -- 172.21.15.27:0/1880678967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 msgr2=0x55fc2e2037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc9b969740 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400bc90 tx=0x7fcc8400f730 comp rx=0 tx=0).stop 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.501+0000 7fcc91400640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 0x7fcc780a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55fc2e1ff510 tx=0x7fcc80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 -- 172.21.15.27:0/1880678967 shutdown_connections 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e2062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e2037b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 --2- 172.21.15.27:0/1880678967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e1fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 -- 172.21.15.27:0/1880678967 >> 172.21.15.27:0/1880678967 conn(0x55fc2e192d10 msgr2=0x55fc2e1f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 -- 172.21.15.27:0/1880678967 shutdown_connections 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 -- 172.21.15.27:0/1880678967 wait complete. 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.502+0000 7fcc9b969740 1 -- start start 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 Processor -- start 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e20fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e206dd0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e0e9f70 con 0x55fc2e203e20 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e206f50 con 0x55fc2e1fe2c0 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc9b969740 10 client.0 _ll_register_callbacks cb 0x55fc2e158480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.503+0000 7fcc91400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53908/0 (socket says 172.21.15.27:53908) 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 -- 172.21.15.27:0/2683074755 learned_addr learned my addr 172.21.15.27:0/2683074755 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc92800640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e20fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91e00640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 -- 172.21.15.27:0/2683074755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 msgr2=0x55fc2e208770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 -- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 msgr2=0x55fc2e20fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e20fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 -- 172.21.15.27:0/2683074755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2df3e6b0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91e00640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc91400640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcc80010cb0 tx=0x7fcc8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd79023ff124da2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.504+0000 7fcc73400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80021420 con 0x55fc2e1fecb0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcc800215c0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc840035f0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc800217c0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc9b969740 10 client.15324 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc2e212da0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.505+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc840035f0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc80029890 con 0x55fc2e1fecb0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 20 client.15324 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e2154e0 0x55fc2e216950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 4 client.15324 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 --> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc2e170210 con 0x55fc2e2154e0 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.506+0000 7fcc91e00640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e2154e0 0x55fc2e216950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.507+0000 7fcc91e00640 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e2154e0 0x55fc2e216950 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb22aacd556f550c server_cookie=9e6861cf343be283 in_seq=0 out_seq=0 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.507+0000 7fcc73400640 10 client.15324 ms_handle_connect on v2:172.21.15.94:6838/12233135 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.509+0000 7fcc73400640 1 -- 172.21.15.27:0/2683074755 <== mds.0 v2:172.21.15.94:6838/12233135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc2e170210 con 0x55fc2e2154e0 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.509+0000 7fcc73400640 10 client.15324 handle_command_reply: tid=0 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 client.15324 shutdown 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 msgr2=0x55fc2e208cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcc80010cb0 tx=0x7fcc8000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 shutdown_connections 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e2154e0 0x55fc2e216950 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e203e20 0x55fc2e20fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e208cb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 --2- 172.21.15.27:0/2683074755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 >> 172.21.15.27:0/2683074755 conn(0x55fc2e192d10 msgr2=0x55fc2e1f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc73400640 0 client.15324 ms_handle_reset on v2:172.21.15.94:6838/12233135 2024-04-26T21:12:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 shutdown_connections 2024-04-26T21:12:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 1 -- 172.21.15.27:0/2683074755 wait complete. 2024-04-26T21:12:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.510+0000 7fcc9b969740 20 client.15324 trim_cache size 0 max 16384 2024-04-26T21:12:02.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 Processor -- start 2024-04-26T21:12:02.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 -- start start 2024-04-26T21:12:02.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e214c90 con 0x55fc2e203e20 2024-04-26T21:12:02.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e215360 con 0x55fc2e1fe2c0 2024-04-26T21:12:02.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.530+0000 7fcc9b969740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e215a30 con 0x55fc2e1fecb0 2024-04-26T21:12:02.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53924/0 (socket says 172.21.15.27:53924) 2024-04-26T21:12:02.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 -- 172.21.15.27:0/363100015 learned_addr learned my addr 172.21.15.27:0/363100015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc91400640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 -- 172.21.15.27:0/363100015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fecb0 msgr2=0x55fc2e20fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc91e00640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 -- 172.21.15.27:0/363100015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 msgr2=0x55fc2e208c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc92800640 1 -- 172.21.15.27:0/363100015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2e1ff540 con 0x55fc2e203e20 2024-04-26T21:12:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.531+0000 7fcc91e00640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc92800640 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0ef580 tx=0x7fcc8c0bd8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092fcc5e1d76a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc7e600640 1 -- 172.21.15.27:0/363100015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0d7070 con 0x55fc2e203e20 2024-04-26T21:12:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc7e600640 1 -- 172.21.15.27:0/363100015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c036040 con 0x55fc2e203e20 2024-04-26T21:12:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc7e600640 1 -- 172.21.15.27:0/363100015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0bbe30 con 0x55fc2e203e20 2024-04-26T21:12:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc9b969740 1 -- 172.21.15.27:0/363100015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 msgr2=0x55fc2e207a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.532+0000 7fcc9b969740 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0ef580 tx=0x7fcc8c0bd8e0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 -- 172.21.15.27:0/363100015 shutdown_connections 2024-04-26T21:12:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e203e20 0x55fc2e207a90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 --2- 172.21.15.27:0/363100015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e208c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 -- 172.21.15.27:0/363100015 >> 172.21.15.27:0/363100015 conn(0x55fc2e192d10 msgr2=0x55fc2e207380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 -- 172.21.15.27:0/363100015 shutdown_connections 2024-04-26T21:12:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 -- 172.21.15.27:0/363100015 wait complete. 2024-04-26T21:12:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 -- start start 2024-04-26T21:12:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.533+0000 7fcc9b969740 1 Processor -- start 2024-04-26T21:12:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e20b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 0x55fc2e20ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e2167d0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e216100 con 0x55fc2e1fe2c0 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc2e216ea0 con 0x55fc2e203e20 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc9b969740 10 client.0 _ll_register_callbacks cb 0x7fcc80012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53936/0 (socket says 172.21.15.27:53936) 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 -- 172.21.15.27:0/48637289 learned_addr learned my addr 172.21.15.27:0/48637289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91e00640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e20b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc92800640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 0x55fc2e20ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 -- 172.21.15.27:0/48637289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 msgr2=0x55fc2e20ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 0x55fc2e20ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 -- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 msgr2=0x55fc2e20b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e20b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.534+0000 7fcc91400640 1 -- 172.21.15.27:0/48637289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2e20f260 con 0x55fc2e1fecb0 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc92800640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 0x55fc2e20ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc91400640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc80029ec0 tx=0x7fcc80012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3746bf07319126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc73400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-26T21:12:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80021420 con 0x55fc2e1fecb0 2024-04-26T21:12:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8c036040 con 0x55fc2e1fecb0 2024-04-26T21:12:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcc800215c0 con 0x55fc2e1fecb0 2024-04-26T21:12:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc80021960 con 0x55fc2e1fecb0 2024-04-26T21:12:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.535+0000 7fcc9b969740 10 client.15330 fetch_fsmap learned FSMap version 27 2024-04-26T21:12:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc2e27b780 con 0x55fc2e1fecb0 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c036040 con 0x55fc2e1fecb0 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc80033610 con 0x55fc2e1fecb0 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 20 client.15330 populate_metadata read hostname 'smithi027' 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e280300 0x55fc2e27ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 4 client.15330 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.536+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 --> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] -- command(tid 0: {"prefix": "status"}) -- 0x55fc2e16a080 con 0x55fc2e280300 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.537+0000 7fcc91e00640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e280300 0x55fc2e27ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc91e00640 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e280300 0x55fc2e27ce10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245095f6bc09254d server_cookie=6dbb485b8e2b75c0 in_seq=0 out_seq=0 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc73400640 10 client.15330 ms_handle_connect on v2:172.21.15.94:6838/12233135 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc73400640 1 -- 172.21.15.27:0/48637289 <== mds.0 v2:172.21.15.94:6838/12233135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fc2e16a080 con 0x55fc2e280300 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc73400640 10 client.15330 handle_command_reply: tid=0 2024-04-26T21:12:02.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 client.15330 shutdown 2024-04-26T21:12:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 msgr2=0x55fc2e20bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc80029ec0 tx=0x7fcc80012dc0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 shutdown_connections 2024-04-26T21:12:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:6838/12233135,v1:172.21.15.94:6839/12233135] conn(0x55fc2e280300 0x55fc2e27ce10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc2e203e20 0x55fc2e20ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc2e1fecb0 0x55fc2e20bc50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 --2- 172.21.15.27:0/48637289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc2e1fe2c0 0x55fc2e20b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc73400640 0 client.15330 ms_handle_reset on v2:172.21.15.94:6838/12233135 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.538+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 >> 172.21.15.27:0/48637289 conn(0x55fc2e192d10 msgr2=0x55fc2e207c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 shutdown_connections 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc9b969740 1 -- 172.21.15.27:0/48637289 wait complete. 2024-04-26T21:12:02.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc9b969740 20 client.15330 trim_cache size 0 max 16384 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "a125c556-040f-11ef-bc93-c7b262605968", 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 34517, 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-26T21:12:02.563 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 140.75746532599999, 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 27, 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 146.396520146 2024-04-26T21:12:02.564 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:02.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 msgr2=0x7fcc780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 0x7fcc780a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55fc2e1ff510 tx=0x7fcc80005850 comp rx=0 tx=0).stop 2024-04-26T21:12:02.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 msgr2=0x7fcc941a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.539+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c031040 tx=0x7fcc8c003ba0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 shutdown_connections 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fcc780a3c40 0x7fcc780a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc9410a940 0x7fcc941a63f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc94109f80 0x7fcc9419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 --2- 172.21.15.27:0/1492971424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc94108d90 0x7fcc9419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 >> 172.21.15.27:0/1492971424 conn(0x7fcc94077d30 msgr2=0x7fcc9410d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 shutdown_connections 2024-04-26T21:12:02.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.540+0000 7fcc93200640 1 -- 172.21.15.27:0/1492971424 wait complete. 2024-04-26T21:12:02.567 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:12:02.577 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:12:02.577 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:12:02.577 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:12:02.578 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T21:12:02.578 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:12:02.578 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:12:02.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T21:12:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.776+0000 7fec2da00640 1 -- 172.21.15.27:0/148296336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 msgr2=0x7fec28106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.776+0000 7fec2da00640 1 --2- 172.21.15.27:0/148296336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec28106060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec10009900 tx=0x7fec1002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 -- 172.21.15.27:0/148296336 shutdown_connections 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 --2- 172.21.15.27:0/148296336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 0x7fec2810ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 --2- 172.21.15.27:0/148296336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec28106060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 --2- 172.21.15.27:0/148296336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec28104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 -- 172.21.15.27:0/148296336 >> 172.21.15.27:0/148296336 conn(0x7fec280fffa0 msgr2=0x7fec281023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 -- 172.21.15.27:0/148296336 shutdown_connections 2024-04-26T21:12:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.777+0000 7fec2da00640 1 -- 172.21.15.27:0/148296336 wait complete. 2024-04-26T21:12:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.778+0000 7fec2da00640 1 Processor -- start 2024-04-26T21:12:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.778+0000 7fec2da00640 1 -- start start 2024-04-26T21:12:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec2819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 0x7fec281a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec28110400 con 0x7fec28105be0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec28110280 con 0x7fec281049f0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec28110580 con 0x7fec281065a0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53966/0 (socket says 172.21.15.27:53966) 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 -- 172.21.15.27:0/1023935118 learned_addr learned my addr 172.21.15.27:0/1023935118 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec27e00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec2819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 -- 172.21.15.27:0/1023935118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 msgr2=0x7fec281a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec2d000640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 0x7fec281a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 0x7fec281a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 -- 172.21.15.27:0/1023935118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 msgr2=0x7fec2819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec2819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.779+0000 7fec1fe00640 1 -- 172.21.15.27:0/1023935118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec281a26f0 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec27e00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec2819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec1fe00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec10031460 tx=0x7fec10003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1003f070 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec10031da0 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec281a2980 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10035030 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.780+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec281a2e70 con 0x7fec28105be0 2024-04-26T21:12:02.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.782+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2810bde0 con 0x7fec28105be0 2024-04-26T21:12:02.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.784+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec10042070 con 0x7fec28105be0 2024-04-26T21:12:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.785+0000 7fec25600640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 0x7fec0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.785+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fec100f0f70 con 0x7fec28105be0 2024-04-26T21:12:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.786+0000 7fec27e00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 0x7fec0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.786+0000 7fec27e00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 0x7fec0c0a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec28105a40 tx=0x7fec18005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:02.789+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fec100bd7e0 con 0x7fec28105be0 2024-04-26T21:12:03.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.049+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fec2802dcf0 con 0x7fec28105be0 2024-04-26T21:12:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.225+0000 7fec25600640 1 -- 172.21.15.27:0/1023935118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7fec100bd0e0 con 0x7fec28105be0 2024-04-26T21:12:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.229+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 msgr2=0x7fec0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.229+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 0x7fec0c0a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec28105a40 tx=0x7fec18005e70 comp rx=0 tx=0).stop 2024-04-26T21:12:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.229+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 msgr2=0x7fec2819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.229+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec10031460 tx=0x7fec10003c80 comp rx=0 tx=0).stop 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 shutdown_connections 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fec0c0a3c30 0x7fec0c0a60f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec281065a0 0x7fec281a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28105be0 0x7fec2819b010 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 --2- 172.21.15.27:0/1023935118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec281049f0 0x7fec2819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 >> 172.21.15.27:0/1023935118 conn(0x7fec280fffa0 msgr2=0x7fec28108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 shutdown_connections 2024-04-26T21:12:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.230+0000 7fec2da00640 1 -- 172.21.15.27:0/1023935118 wait complete. 2024-04-26T21:12:03.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T21:12:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 -- 172.21.15.27:0/1162152489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 msgr2=0x7f64f410ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 --2- 172.21.15.27:0/1162152489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f410ae00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009ca0 tx=0x7f64ec02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 -- 172.21.15.27:0/1162152489 shutdown_connections 2024-04-26T21:12:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 --2- 172.21.15.27:0/1162152489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f410ae00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 --2- 172.21.15.27:0/1162152489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 0x7f64f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 --2- 172.21.15.27:0/1162152489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 -- 172.21.15.27:0/1162152489 >> 172.21.15.27:0/1162152489 conn(0x7f64f40fdf50 msgr2=0x7f64f4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 -- 172.21.15.27:0/1162152489 shutdown_connections 2024-04-26T21:12:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.448+0000 7f64fbc00640 1 -- 172.21.15.27:0/1162152489 wait complete. 2024-04-26T21:12:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 Processor -- start 2024-04-26T21:12:03.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 -- start start 2024-04-26T21:12:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 0x7f64f4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410e3f0 con 0x7f64f4104570 2024-04-26T21:12:03.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410e270 con 0x7f64f4103bb0 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.449+0000 7f64fbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410e570 con 0x7f64f41029c0 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54012/0 (socket says 172.21.15.27:54012) 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 -- 172.21.15.27:0/3487060000 learned_addr learned my addr 172.21.15.27:0/3487060000 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f3e00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 0x7f64f4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 msgr2=0x7f64f4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 msgr2=0x7f64f4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 0x7f64f4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9c00640 1 -- 172.21.15.27:0/3487060000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f41a05e0 con 0x7f64f4104570 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.450+0000 7f64f9200640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64f9c00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005e30 tx=0x7f64ec003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec003d60 con 0x7f64f4104570 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f4074b60 con 0x7f64f4104570 2024-04-26T21:12:03.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64ec0044d0 con 0x7f64f4104570 2024-04-26T21:12:03.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec030ba0 con 0x7f64f4104570 2024-04-26T21:12:03.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.451+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f4075100 con 0x7f64f4104570 2024-04-26T21:12:03.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.452+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f4109c80 con 0x7f64f4104570 2024-04-26T21:12:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.454+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f64ec004680 con 0x7f64f4104570 2024-04-26T21:12:03.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.456+0000 7f64f1600640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 0x7f64dc0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.456+0000 7f64f9200640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 msgr2=0x7f64dc0a6200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.456+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f64ec0f1960 con 0x7f64f4104570 2024-04-26T21:12:03.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.456+0000 7f64f9200640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 0x7f64dc0a6200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:12:03.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.465+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64ec0be2e0 con 0x7f64f4104570 2024-04-26T21:12:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-mon[28085]: pgmap v440: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:12:03.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T21:12:03.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setuser ceph since I am not root 2024-04-26T21:12:03.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: ignoring --setgroup ceph since I am not root 2024-04-26T21:12:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-mon[32145]: pgmap v440: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:12:03.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T21:12:03.503 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: ignoring --setuser ceph since I am not root 2024-04-26T21:12:03.503 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: ignoring --setgroup ceph since I am not root 2024-04-26T21:12:03.503 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-mgr[33898]: -- 172.21.15.204:0/1010726106 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a0588a4e0 con 0x557a05861000 2024-04-26T21:12:03.503 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-mgr[33898]: -- 172.21.15.204:0/1010726106 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a0588c5a0 con 0x557a05861000 2024-04-26T21:12:03.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: ignoring --setuser ceph since I am not root 2024-04-26T21:12:03.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: ignoring --setgroup ceph since I am not root 2024-04-26T21:12:03.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-mgr[32663]: -- 172.21.15.94:0/369114697 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559636bde5a0 con 0x559636bb3000 2024-04-26T21:12:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-mon[31273]: pgmap v440: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:12:03.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T21:12:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.657+0000 7f64f9200640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 msgr2=0x7f64dc0a6200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.657+0000 7f64f9200640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 0x7f64dc0a6200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:12:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.670+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f64f40764e0 con 0x7f64f4104570 2024-04-26T21:12:03.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.676+0000 7f64f1600640 1 -- 172.21.15.27:0/3487060000 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f64ec0bdbe0 con 0x7f64f4104570 2024-04-26T21:12:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 msgr2=0x7f64dc0a6200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 0x7f64dc0a6200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 msgr2=0x7f64f419ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005e30 tx=0x7f64ec003970 comp rx=0 tx=0).stop 2024-04-26T21:12:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 shutdown_connections 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f64dc0a3d40 0x7f64dc0a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64f4104570 0x7f64f419ff70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64f4103bb0 0x7f64f4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 --2- 172.21.15.27:0/3487060000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f41029c0 0x7f64f4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 >> 172.21.15.27:0/3487060000 conn(0x7f64f40fdf50 msgr2=0x7f64f4106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 shutdown_connections 2024-04-26T21:12:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.679+0000 7f64fbc00640 1 -- 172.21.15.27:0/3487060000 wait complete. 2024-04-26T21:12:03.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T21:12:03.825 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:03.567+0000 7ff1bb7eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:12:03.848 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:03.589+0000 7f4738b1b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:12:03.848 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:03.845+0000 7f4738b1b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 -- 172.21.15.27:0/274781346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 msgr2=0x7f2548109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 --2- 172.21.15.27:0/274781346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f2548109af0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f254402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 -- 172.21.15.27:0/274781346 shutdown_connections 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 --2- 172.21.15.27:0/274781346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f2548109af0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 --2- 172.21.15.27:0/274781346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 0x7f2548102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 --2- 172.21.15.27:0/274781346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 0x7f2548101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 -- 172.21.15.27:0/274781346 >> 172.21.15.27:0/274781346 conn(0x7f25480fcc20 msgr2=0x7f25480ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 -- 172.21.15.27:0/274781346 shutdown_connections 2024-04-26T21:12:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.890+0000 7f254f200640 1 -- 172.21.15.27:0/274781346 wait complete. 2024-04-26T21:12:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.891+0000 7f254f200640 1 Processor -- start 2024-04-26T21:12:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.891+0000 7f254f200640 1 -- start start 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 0x7f254810bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 0x7f254810a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f254810e210 con 0x7f2548103260 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f254810e090 con 0x7f25481028a0 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f254810e390 con 0x7f25481016b0 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54036/0 (socket says 172.21.15.27:54036) 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 -- 172.21.15.27:0/1405840976 learned_addr learned my addr 172.21.15.27:0/1405840976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254de00640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 0x7f254810bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254d400640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 0x7f254810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 msgr2=0x7f254810bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 0x7f254810bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 msgr2=0x7f254810a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 0x7f254810a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.892+0000 7f254e800640 1 -- 172.21.15.27:0/1405840976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254810b470 con 0x7f2548103260 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f254e800640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2544031040 tx=0x7f2544003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544003d40 con 0x7f2548103260 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25481a29f0 con 0x7f2548103260 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25440044d0 con 0x7f2548103260 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544030c30 con 0x7f2548103260 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.893+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a2f60 con 0x7f2548103260 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.895+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2548108b10 con 0x7f2548103260 2024-04-26T21:12:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.898+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f2544004040 con 0x7f2548103260 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.899+0000 7f2542a00640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 0x7f25300a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.899+0000 7f254de00640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 msgr2=0x7f25300a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.899+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f25440f1b20 con 0x7f2548103260 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.899+0000 7f254de00640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 0x7f25300a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:12:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:03.909+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f25440be4a0 con 0x7f2548103260 2024-04-26T21:12:03.955 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:03.579+0000 7f204f21b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:12:03.955 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:03.821+0000 7f204f21b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:03.955 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:03.953+0000 7f204f21b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:12:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.100+0000 7f254de00640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 msgr2=0x7f25300a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.100+0000 7f254de00640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 0x7f25300a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:12:04.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.116+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f25480635a0 con 0x7f2548103260 2024-04-26T21:12:04.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.124+0000 7f2542a00640 1 -- 172.21.15.27:0/1405840976 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f2544039e10 con 0x7f2548103260 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 msgr2=0x7f25300a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 0x7f25300a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 msgr2=0x7f254810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2544031040 tx=0x7f2544003ba0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 shutdown_connections 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7f25300a3c80 0x7f25300a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2548103260 0x7f254810abb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25481028a0 0x7f254810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.126+0000 7f254f200640 1 --2- 172.21.15.27:0/1405840976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25481016b0 0x7f254810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.127+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 >> 172.21.15.27:0/1405840976 conn(0x7f25480fcc20 msgr2=0x7f2548105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.127+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 shutdown_connections 2024-04-26T21:12:04.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:04.127+0000 7f254f200640 1 -- 172.21.15.27:0/1405840976 wait complete. 2024-04-26T21:12:04.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:03.824+0000 7ff1bb7eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:04.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:03.972+0000 7ff1bb7eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:12:04.146 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T21:12:04.171 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T21:12:04.171 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-26T21:12:04.171 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:12:04.172 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:12:04.172 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:04.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:04.172 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T21:12:04.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:04.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-26T21:12:04.229 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:03.986+0000 7f4738b1b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:12:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-04-26T21:12:04.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:12:04.312 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T21:12:04.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:12:04.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T21:12:04.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-mon[28085]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-04-26T21:12:04.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3487060000' entity='client.admin' 2024-04-26T21:12:04.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1405840976' entity='client.admin' 2024-04-26T21:12:04.494 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:04.377+0000 7f4738b1b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:12:04.494 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T21:12:04.494 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-mon[31273]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-04-26T21:12:04.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3487060000' entity='client.admin' 2024-04-26T21:12:04.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1405840976' entity='client.admin' 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:04.349+0000 7f204f21b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:04.464+0000 7f204f21b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1023935118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-mon[32145]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3487060000' entity='client.admin' 2024-04-26T21:12:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:04 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1405840976' entity='client.admin' 2024-04-26T21:12:04.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:04.400+0000 7ff1bb7eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:12:04.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:04.521+0000 7ff1bb7eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:12:04.754 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:12:04.775 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:04.492+0000 7f4738b1b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:12:06.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:05.988+0000 7f204f21b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:12:06.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:06.212+0000 7f204f21b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:06.425 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:06.176+0000 7ff1bb7eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:12:06.471 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:06.027+0000 7f4738b1b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:12:06.471 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:06.253+0000 7f4738b1b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:06.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:06.423+0000 7ff1bb7eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:06.735 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:06.656+0000 7ff1bb7eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:12:06.751 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:06.428+0000 7f204f21b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:12:06.775 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:06.469+0000 7f4738b1b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:12:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 -- 172.21.15.27:0/3693781166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 msgr2=0x7fb2e0108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/3693781166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e0108790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00099c0 tx=0x7fb2d0009cd0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 -- 172.21.15.27:0/3693781166 shutdown_connections 2024-04-26T21:12:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/3693781166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e0103e90 0x7fb2e0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/3693781166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e0108790 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/3693781166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e01034d0 0x7fb2e0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 -- 172.21.15.27:0/3693781166 >> 172.21.15.27:0/3693781166 conn(0x7fb2e007a580 msgr2=0x7fb2e007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 -- 172.21.15.27:0/3693781166 shutdown_connections 2024-04-26T21:12:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.911+0000 7fb2e4c00700 1 -- 172.21.15.27:0/3693781166 wait complete. 2024-04-26T21:12:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.912+0000 7fb2e4c00700 1 Processor -- start 2024-04-26T21:12:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.912+0000 7fb2e4c00700 1 -- start start 2024-04-26T21:12:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e01034d0 0x7fb2e0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 0x7fb2e0075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0113510 con 0x7fb2e01083b0 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0113390 con 0x7fb2e01034d0 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2e4c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0113690 con 0x7fb2e0103e90 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2de800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2de800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54058/0 (socket says 172.21.15.27:54058) 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2de800700 1 -- 172.21.15.27:0/1544675714 learned_addr learned my addr 172.21.15.27:0/1544675714 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2dde00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e01034d0 0x7fb2e0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.913+0000 7fb2dd400700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 0x7fb2e0075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 msgr2=0x7fb2e0075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 0x7fb2e0075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e01034d0 msgr2=0x7fb2e0075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e01034d0 0x7fb2e0075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 -- 172.21.15.27:0/1544675714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e01a7f10 con 0x7fb2e01083b0 2024-04-26T21:12:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2dd400700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 0x7fb2e0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2de800700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800cca0 tx=0x7fb2d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8016020 con 0x7fb2e01083b0 2024-04-26T21:12:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2d800aba0 con 0x7fb2e01083b0 2024-04-26T21:12:06.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80115e0 con 0x7fb2e01083b0 2024-04-26T21:12:06.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.914+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e01a8200 con 0x7fb2e01083b0 2024-04-26T21:12:06.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.915+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e01a86f0 con 0x7fb2e01083b0 2024-04-26T21:12:06.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.916+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e006a110 con 0x7fb2e01083b0 2024-04-26T21:12:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.918+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7fb2d8019020 con 0x7fb2e01083b0 2024-04-26T21:12:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.919+0000 7fb2d6a00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 0x7fb2c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.919+0000 7fb2dde00700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 msgr2=0x7fb2c40a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.919+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb2d80d0120 con 0x7fb2e01083b0 2024-04-26T21:12:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.920+0000 7fb2dde00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 0x7fb2c40a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:12:06.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:06.928+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2d8020030 con 0x7fb2e01083b0 2024-04-26T21:12:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.121+0000 7fb2dde00700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 msgr2=0x7fb2c40a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3365757774 2024-04-26T21:12:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.121+0000 7fb2dde00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 0x7fb2c40a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:12:07.185 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:06.805+0000 7f204f21b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:12:07.185 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:06.929+0000 7f204f21b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:12:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.206+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb2e002e490 con 0x7fb2e01083b0 2024-04-26T21:12:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.207+0000 7fb2d6a00700 1 -- 172.21.15.27:0/1544675714 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fb2d809caa0 con 0x7fb2e01083b0 2024-04-26T21:12:07.208 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:12:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 msgr2=0x7fb2c40a6260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 0x7fb2c40a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 msgr2=0x7fb2e00761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800cca0 tx=0x7fb2d800a060 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 shutdown_connections 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:6800/3365757774,v1:172.21.15.27:6801/3365757774] conn(0x7fb2c40a3da0 0x7fb2c40a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2e01083b0 0x7fb2e00761a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e01034d0 0x7fb2e0075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 --2- 172.21.15.27:0/1544675714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e0103e90 0x7fb2e0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 >> 172.21.15.27:0/1544675714 conn(0x7fb2e007a580 msgr2=0x7fb2e01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.210+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 shutdown_connections 2024-04-26T21:12:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:07.211+0000 7fb2e4c00700 1 -- 172.21.15.27:0/1544675714 wait complete. 2024-04-26T21:12:07.224 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:06.846+0000 7f4738b1b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:12:07.224 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:06.969+0000 7f4738b1b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:12:07.441 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:07.183+0000 7f204f21b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:12:07.441 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:07 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1544675714' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:12:07.469 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:07 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1544675714' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:12:07.470 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:07.065+0000 7ff1bb7eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:12:07.470 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:07.197+0000 7ff1bb7eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:12:07.470 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:07.468+0000 7ff1bb7eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:12:07.478 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:07 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1544675714' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:12:07.479 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:07.221+0000 7f4738b1b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:12:07.752 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:07.440+0000 7f204f21b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:07.775 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:07.476+0000 7f4738b1b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:08.046 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:07.738+0000 7ff1bb7eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:12:08.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:12:08.092 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:12:08.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:08.092 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-04-26T21:12:08.142 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-04-26T21:12:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41809sec preferred_lft 41809sec 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-04-26T21:12:08.145 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-04-26T21:12:08.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:12:08.146 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T21:12:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:08.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:08.228+0000 7f204f21b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:12:08.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:08.267+0000 7f4738b1b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:12:08.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:08.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:08.648 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:12:08.715 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-04-26T21:12:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:08.718 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-04-26T21:12:08.773 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-04-26T21:12:08.773 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:12:08.773 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:12:08.774 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:12:08.774 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:12:08.774 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T21:12:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:08.897 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:08.595+0000 7ff1bb7eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:12:09.181 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:08.782+0000 7f204f21b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:12:09.181 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:08.915+0000 7f204f21b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:12:09.206 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:08.814+0000 7f4738b1b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:12:09.206 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:08.943+0000 7f4738b1b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:12:09.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:09.207+0000 7ff1bb7eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:12:09.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:09.347+0000 7ff1bb7eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:12:09.501 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:09.180+0000 7f204f21b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:12:09.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:09.204+0000 7f4738b1b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:12:09.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:09.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:09.555 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T21:12:09.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:09.601 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-04-26T21:12:09.693 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:09.693 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T21:12:09.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:09.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:12:09.693 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T21:12:09.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:09.818 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:09.632+0000 7ff1bb7eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:12:09.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:09.873 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:12:09.873 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T21:12:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:10.172 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:10.172 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-26T21:12:10.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-04-26T21:12:10.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:12:10.172 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-26T21:12:10.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:09.830+0000 7f204f21b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:12:10.252 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:09.951+0000 7f204f21b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:12:10.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:10.276 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:09.846+0000 7f4738b1b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:12:10.276 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:09.970+0000 7f4738b1b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:12:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:10.369 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:12:10.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:10.369 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:10.411 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:12:10.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:10.411 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:10.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T21:12:10.584 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:10.275+0000 7f204f21b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:12:10.584 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:10.455+0000 7f204f21b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:12:10.584 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:10.583+0000 7f204f21b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:12:10.591 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:12:10.606 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:10.300+0000 7f4738b1b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:12:10.606 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:10.477+0000 7f4738b1b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:12:10.644 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T21:12:10.645 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3c22637e6b8037da4201dc3e1659e7d921827de307bc9567359243ba59d303ce/merged 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/29ae3f6cdf1eb033c3c015d4f61329c7927845e09cb779db63b695323482bb6a/merged 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T21:12:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/43ea67303118061256e513f5d3692060c2ae56162f3ba2083839bead6d7323df/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6775ee521d582704fcd081eb0fd9bf4f3b72d0d90f7f7eab8be3ef15ad0b896e/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/958f18fd6f6cbb6b03a838c9565e664be91b8b2fc22adb67ea6222b44517498c/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f133b5a4a3720acda71d0ef2cc2196ac343c7e298bba5004becad19029867c55/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/16863443766d9c4fe3f0b41b798721ecad1a69a35ac572e977eea6076c70a697/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b0a43383f0b30ce2b525e4bcefe710b6a8c11e0d0e1fbf53e5ec1854b7281645/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e35bdf526f62ab8122bd43775250a3b607a9a29608043d4cfaf4bab3c477ff5a/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/11ad448a9d83101def4d2f1a25271b30e08c0e88efdeac90ada122826ae9ece5/merged 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:10.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:10.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:10.648 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T21:12:10.694 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:12:10.694 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/4f65e73b-e35b-4043-a5ec-4c24d8b66b8e --client_fs=cephfs) 2024-04-26T21:12:10.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T21:12:10.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:10.341+0000 7ff1bb7eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:12:10.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:10.476+0000 7ff1bb7eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:12:10.764 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3c22637e6b8037da4201dc3e1659e7d921827de307bc9567359243ba59d303ce/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/29ae3f6cdf1eb033c3c015d4f61329c7927845e09cb779db63b695323482bb6a/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/43ea67303118061256e513f5d3692060c2ae56162f3ba2083839bead6d7323df/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6775ee521d582704fcd081eb0fd9bf4f3b72d0d90f7f7eab8be3ef15ad0b896e/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/958f18fd6f6cbb6b03a838c9565e664be91b8b2fc22adb67ea6222b44517498c/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f133b5a4a3720acda71d0ef2cc2196ac343c7e298bba5004becad19029867c55/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/16863443766d9c4fe3f0b41b798721ecad1a69a35ac572e977eea6076c70a697/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b0a43383f0b30ce2b525e4bcefe710b6a8c11e0d0e1fbf53e5ec1854b7281645/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e35bdf526f62ab8122bd43775250a3b607a9a29608043d4cfaf4bab3c477ff5a/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/11ad448a9d83101def4d2f1a25271b30e08c0e88efdeac90ada122826ae9ece5/merged 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:10.855 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T21:12:10.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2024-04-26T21:12:10.886+0000 7fdeaa8ac480 -1 init, newargv = 0x5575a5a2a2d0 newargc=15 2024-04-26T21:12:10.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[80721]: starting ceph client 2024-04-26T21:12:10.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[80721]: starting fuse 2024-04-26T21:12:11.001 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:10.733+0000 7f204f21b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:11.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:10.604+0000 7f4738b1b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:12:11.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:10.755+0000 7f4738b1b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:11.071 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:10.872+0000 7ff1bb7eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:12:11.071 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:11.069+0000 7ff1bb7eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:12:11.368 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:11.205+0000 7ff1bb7eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:12:11.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:11.107+0000 7f204f21b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:12:11.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:11.132+0000 7f4738b1b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:12:11.734 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:11.365+0000 7ff1bb7eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:12:12.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-26T21:12:12.096 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:11.756+0000 7ff1bb7eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:12:12.109 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-26T21:12:12.156 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:12:12.157 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3c22637e6b8037da4201dc3e1659e7d921827de307bc9567359243ba59d303ce/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/29ae3f6cdf1eb033c3c015d4f61329c7927845e09cb779db63b695323482bb6a/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/43ea67303118061256e513f5d3692060c2ae56162f3ba2083839bead6d7323df/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6775ee521d582704fcd081eb0fd9bf4f3b72d0d90f7f7eab8be3ef15ad0b896e/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/958f18fd6f6cbb6b03a838c9565e664be91b8b2fc22adb67ea6222b44517498c/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f133b5a4a3720acda71d0ef2cc2196ac343c7e298bba5004becad19029867c55/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/16863443766d9c4fe3f0b41b798721ecad1a69a35ac572e977eea6076c70a697/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/b0a43383f0b30ce2b525e4bcefe710b6a8c11e0d0e1fbf53e5ec1854b7281645/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/e35bdf526f62ab8122bd43775250a3b607a9a29608043d4cfaf4bab3c477ff5a/merged 2024-04-26T21:12:12.158 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/11ad448a9d83101def4d2f1a25271b30e08c0e88efdeac90ada122826ae9ece5/merged 2024-04-26T21:12:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-26T21:12:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:12.159 INFO:teuthology.orchestra.run.smithi027.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:12.159 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:11.866+0000 7f204f21b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:12:12.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:12.160 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-26T21:12:12.171 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:11.884+0000 7f4738b1b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:12:12.209 INFO:teuthology.orchestra.run.smithi027.stdout:114 2024-04-26T21:12:12.209 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-04-26T21:12:12.210 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2024-04-26T21:12:12.211 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-04-26T21:12:12.365 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.80721.asok 2024-04-26T21:12:12.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: Standby manager daemon z restarted 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: Standby manager daemon z started 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: Standby manager daemon y restarted 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: Standby manager daemon y started 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:12:12.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-mon[28085]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.382 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80721.asok 2024-04-26T21:12:12.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:12.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80721.asok status 2024-04-26T21:12:12.501 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z[33894]: 2024-04-26T21:12:12.157+0000 7f204f21b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: Standby manager daemon z restarted 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: Standby manager daemon z started 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: Standby manager daemon y restarted 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: Standby manager daemon y started 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:12:12.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:12 smithi204 ceph-mon[32145]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.525 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y[32659]: 2024-04-26T21:12:12.169+0000 7f4738b1b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:12:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: Standby manager daemon z restarted 2024-04-26T21:12:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: Standby manager daemon z started 2024-04-26T21:12:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.204:0/3095446013' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: Standby manager daemon y restarted 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: Standby manager daemon y started 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:12:12.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:12 smithi094 ceph-mon[31273]: from='mgr.? 172.21.15.94:0/2142230575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:12:12.602 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:12:12.602 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "80721", 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/volumes/qa/sv_0/4f65e73b-e35b-4043-a5ec-4c24d8b66b8e" 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 15378, 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-04-26T21:12:12.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 15378 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 208131370 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 208131370 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.15378 192.168.0.1:0/208131370", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/208131370", 2024-04-26T21:12:12.604 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 5, 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 27, 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 90, 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-04-26T21:12:12.605 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:12:12.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:12.617 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-04-26T21:12:12.617 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:12.669 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-04-26T21:12:12.669 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:12.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:12.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:12.728 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:12.582+0000 7ff1bb7eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:12:12.743 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T21:12:12.752 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:12:12.753 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:12:12.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.943+0000 7f5de1600640 1 -- 172.21.15.27:0/3028403404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 msgr2=0x7f5ddc103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.943+0000 7f5de1600640 1 --2- 172.21.15.27:0/3028403404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc103630 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009900 tx=0x7f5dcc02a410 comp rx=0 tx=0).stop 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 -- 172.21.15.27:0/3028403404 shutdown_connections 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 --2- 172.21.15.27:0/3028403404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ddc103b70 0x7f5ddc105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 --2- 172.21.15.27:0/3028403404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc106520 0x7f5ddc1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 --2- 172.21.15.27:0/3028403404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc103630 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 -- 172.21.15.27:0/3028403404 >> 172.21.15.27:0/3028403404 conn(0x7f5ddc0fce00 msgr2=0x7f5ddc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 -- 172.21.15.27:0/3028403404 shutdown_connections 2024-04-26T21:12:12.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.944+0000 7f5de1600640 1 -- 172.21.15.27:0/3028403404 wait complete. 2024-04-26T21:12:12.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.945+0000 7f5de1600640 1 Processor -- start 2024-04-26T21:12:12.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 -- start start 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103b70 0x7f5ddc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ddc106520 0x7f5ddc1a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc114a20 con 0x7f5ddc101200 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1148a0 con 0x7f5ddc103b70 2024-04-26T21:12:12.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.946+0000 7f5de1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc114ba0 con 0x7f5ddc106520 2024-04-26T21:12:12.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.947+0000 7f5ddaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.947+0000 7f5ddaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32814/0 (socket says 172.21.15.27:32814) 2024-04-26T21:12:12.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.947+0000 7f5ddaa00640 1 -- 172.21.15.27:0/2409661338 learned_addr learned my addr 172.21.15.27:0/2409661338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:12.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 -- 172.21.15.27:0/2409661338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ddc106520 msgr2=0x7f5ddc1a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ddc106520 0x7f5ddc1a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 -- 172.21.15.27:0/2409661338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103b70 msgr2=0x7f5ddc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103b70 0x7f5ddc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 -- 172.21.15.27:0/2409661338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a6cc0 con 0x7f5ddc101200 2024-04-26T21:12:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.948+0000 7f5ddaa00640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc031460 tx=0x7f5dcc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.949+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc03f070 con 0x7f5ddc101200 2024-04-26T21:12:12.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.949+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dcc031da0 con 0x7f5ddc101200 2024-04-26T21:12:12.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.949+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a6f50 con 0x7f5ddc101200 2024-04-26T21:12:12.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.949+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc035030 con 0x7f5ddc101200 2024-04-26T21:12:12.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.949+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a7430 con 0x7f5ddc101200 2024-04-26T21:12:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.952+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc0052c0 con 0x7f5ddc101200 2024-04-26T21:12:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.953+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f5dcc042070 con 0x7f5ddc101200 2024-04-26T21:12:12.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.956+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8549+0+0 (secure 0 0 0) 0x7f5dcc0f09b0 con 0x7f5ddc101200 2024-04-26T21:12:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:12.969+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5dcc0bd5f0 con 0x7f5ddc101200 2024-04-26T21:12:12.985 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:12.891+0000 7ff1bb7eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:12:13.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:13.353+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f5dbc0055b0 con 0x7f5ddc101200 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.329+0000 7ff10b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.329+0000 7ff10b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.329+0000 7ff10b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.329+0000 7ff10b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.329+0000 7ff10b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.330+0000 7ff111200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.330+0000 7ff111200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.330+0000 7ff111200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.330+0000 7ff111200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:12:13.330+0000 7ff111200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: Active manager daemon x restarted 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: Activating manager daemon x 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: mgrmap e20: x(active, starting, since 0.0294335s), standbys: z, y 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:12:13.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:12:13.487 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:12:13.488 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:12:13.489 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:12:13.489 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: Manager daemon x is now available 2024-04-26T21:12:13.489 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:13.489 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:12:13.489 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:13 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: Active manager daemon x restarted 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: Activating manager daemon x 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: mgrmap e20: x(active, starting, since 0.0294335s), standbys: z, y 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:12:13.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:12:13.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:12:13.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: Manager daemon x is now available 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:12:13.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:13 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: Active manager daemon x restarted 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: Activating manager daemon x 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: mgrmap e20: x(active, starting, since 0.0294335s), standbys: z, y 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:12:13.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:12:13.528 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:12:13.529 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: Manager daemon x is now available 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:12:13.530 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:13 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:12:13.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:13.933+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f5dcc0b8c40 con 0x7f5ddc101200 2024-04-26T21:12:13.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:13.935+0000 7f5dd3400640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 0x7f5dc00ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:13.938+0000 7f5dda000640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 0x7f5dc00ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:13.942+0000 7f5dda000640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 0x7f5dc00ace10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc1a05b0 tx=0x7f5dc8005bf0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.251+0000 7f5dd3400640 1 -- 172.21.15.27:0/2409661338 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f5dcc0cfa80 con 0x7f5ddc101200 2024-04-26T21:12:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 msgr2=0x7f5dc00ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 0x7f5dc00ace10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc1a05b0 tx=0x7f5dc8005bf0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 msgr2=0x7f5ddc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc031460 tx=0x7f5dcc003d50 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 shutdown_connections 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dc00aaa20 0x7f5dc00ace10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ddc106520 0x7f5ddc1a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103b70 0x7f5ddc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 --2- 172.21.15.27:0/2409661338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ddc101200 0x7f5ddc19f0a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.254+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 >> 172.21.15.27:0/2409661338 conn(0x7f5ddc0fce00 msgr2=0x7f5ddc104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.255+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 shutdown_connections 2024-04-26T21:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.255+0000 7f5de1600640 1 -- 172.21.15.27:0/2409661338 wait complete. 2024-04-26T21:12:14.275 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T21:12:14.287 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T21:12:14.317 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:12:14.317 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:12:14.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T21:12:14.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:14 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:12:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:14 smithi204 ceph-mon[32145]: mgrmap e21: x(active, since 1.04016s), standbys: z, y 2024-04-26T21:12:14.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 -- 172.21.15.27:0/3141816522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84106760 msgr2=0x7eff8410cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 --2- 172.21.15.27:0/3141816522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84106760 0x7eff8410cff0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009ca0 tx=0x7eff7c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 -- 172.21.15.27:0/3141816522 shutdown_connections 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 --2- 172.21.15.27:0/3141816522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 0x7eff84106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 --2- 172.21.15.27:0/3141816522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84106760 0x7eff8410cff0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 --2- 172.21.15.27:0/3141816522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84104bb0 0x7eff84104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 -- 172.21.15.27:0/3141816522 >> 172.21.15.27:0/3141816522 conn(0x7eff84100120 msgr2=0x7eff84102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 -- 172.21.15.27:0/3141816522 shutdown_connections 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.514+0000 7eff89600640 1 -- 172.21.15.27:0/3141816522 wait complete. 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.515+0000 7eff89600640 1 Processor -- start 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.515+0000 7eff89600640 1 -- start start 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 0x7eff8419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84106760 0x7eff841a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff841105c0 con 0x7eff84104bb0 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff84110440 con 0x7eff84106760 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff89600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff84110740 con 0x7eff84105da0 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32926/0 (socket says 172.21.15.27:32926) 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 -- 172.21.15.27:0/2301737734 learned_addr learned my addr 172.21.15.27:0/2301737734 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff83400640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84106760 0x7eff841a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82000640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 0x7eff8419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 -- 172.21.15.27:0/2301737734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 msgr2=0x7eff8419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 0x7eff8419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 -- 172.21.15.27:0/2301737734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84106760 msgr2=0x7eff841a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84106760 0x7eff841a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.516+0000 7eff82a00640 1 -- 172.21.15.27:0/2301737734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841a27f0 con 0x7eff84104bb0 2024-04-26T21:12:14.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff82a00640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e9c0 tx=0x7eff7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7400cd80 con 0x7eff84104bb0 2024-04-26T21:12:14.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff740044b0 con 0x7eff84104bb0 2024-04-26T21:12:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841a2ae0 con 0x7eff84104bb0 2024-04-26T21:12:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74010430 con 0x7eff84104bb0 2024-04-26T21:12:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.517+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841a3080 con 0x7eff84104bb0 2024-04-26T21:12:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.518+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8410bed0 con 0x7eff84104bb0 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.520+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7eff74003f00 con 0x7eff84104bb0 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.523+0000 7eff7b400640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 0x7eff680a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.523+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8549+0+0 (secure 0 0 0) 0x7eff740d0580 con 0x7eff84104bb0 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.523+0000 7eff82000640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 0x7eff680a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.524+0000 7eff82000640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 0x7eff680a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff70005fd0 tx=0x7eff70005850 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:14 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:12:14.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:14 smithi094 ceph-mon[31273]: mgrmap e21: x(active, since 1.04016s), standbys: z, y 2024-04-26T21:12:14.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.527+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff7409d0d0 con 0x7eff84104bb0 2024-04-26T21:12:14.686 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:14 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:12:14.687 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:14 smithi027 ceph-mon[28085]: mgrmap e21: x(active, since 1.04016s), standbys: z, y 2024-04-26T21:12:14.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.759+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7eff84062d70 con 0x7eff680a3ae0 2024-04-26T21:12:14.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.903+0000 7eff7b400640 1 -- 172.21.15.27:0/2301737734 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7eff84062d70 con 0x7eff680a3ae0 2024-04-26T21:12:14.905 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.905+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 msgr2=0x7eff680a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.905+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 0x7eff680a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff70005fd0 tx=0x7eff70005850 comp rx=0 tx=0).stop 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.905+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 msgr2=0x7eff8419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.905+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e9c0 tx=0x7eff7400ee80 comp rx=0 tx=0).stop 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.905+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 shutdown_connections 2024-04-26T21:12:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7eff680a3ae0 0x7eff680a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff84105da0 0x7eff8419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff84106760 0x7eff841a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 --2- 172.21.15.27:0/2301737734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff84104bb0 0x7eff8419abd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 >> 172.21.15.27:0/2301737734 conn(0x7eff84100120 msgr2=0x7eff841090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 shutdown_connections 2024-04-26T21:12:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:14.906+0000 7eff89600640 1 -- 172.21.15.27:0/2301737734 wait complete. 2024-04-26T21:12:14.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.115+0000 7f5deb200640 1 -- 172.21.15.27:0/3131592236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 msgr2=0x7f5dec111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.115+0000 7f5deb200640 1 --2- 172.21.15.27:0/3131592236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec111220 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009ca0 tx=0x7f5de402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 -- 172.21.15.27:0/3131592236 shutdown_connections 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 --2- 172.21.15.27:0/3131592236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec111220 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 --2- 172.21.15.27:0/3131592236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec109fd0 0x7f5dec10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 --2- 172.21.15.27:0/3131592236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec108de0 0x7f5dec109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 -- 172.21.15.27:0/3131592236 >> 172.21.15.27:0/3131592236 conn(0x7f5dec077dc0 msgr2=0x7f5dec07a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 -- 172.21.15.27:0/3131592236 shutdown_connections 2024-04-26T21:12:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.116+0000 7f5deb200640 1 -- 172.21.15.27:0/3131592236 wait complete. 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 Processor -- start 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 -- start start 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec108de0 0x7f5dec19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec109fd0 0x7f5dec19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec114810 con 0x7f5dec10a990 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec114690 con 0x7f5dec108de0 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.117+0000 7f5deb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec114990 con 0x7f5dec109fd0 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32952/0 (socket says 172.21.15.27:32952) 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 -- 172.21.15.27:0/2575753241 learned_addr learned my addr 172.21.15.27:0/2575753241 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5de9e00640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec108de0 0x7f5dec19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5de9400640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec109fd0 0x7f5dec19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 -- 172.21.15.27:0/2575753241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec109fd0 msgr2=0x7f5dec19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec109fd0 0x7f5dec19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 -- 172.21.15.27:0/2575753241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec108de0 msgr2=0x7f5dec19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec108de0 0x7f5dec19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.118+0000 7f5dea800640 1 -- 172.21.15.27:0/2575753241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1a6b00 con 0x7f5dec10a990 2024-04-26T21:12:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5dea800640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005e30 tx=0x7f5de4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4003d60 con 0x7f5dec10a990 2024-04-26T21:12:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5de40044d0 con 0x7f5dec10a990 2024-04-26T21:12:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1a6d90 con 0x7f5dec10a990 2024-04-26T21:12:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4030ba0 con 0x7f5dec10a990 2024-04-26T21:12:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.119+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a7280 con 0x7f5dec10a990 2024-04-26T21:12:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.120+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec1101a0 con 0x7f5dec10a990 2024-04-26T21:12:15.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.122+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f5de4004680 con 0x7f5dec10a990 2024-04-26T21:12:15.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.124+0000 7f5de2a00640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 0x7f5dd00a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.124+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8549+0+0 (secure 0 0 0) 0x7f5de40f1650 con 0x7f5dec10a990 2024-04-26T21:12:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.124+0000 7f5de9e00640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 0x7f5dd00a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.125+0000 7f5de9e00640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 0x7f5dd00a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5dec109e30 tx=0x7f5ddc005f90 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.129+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5de40be1a0 con 0x7f5dec10a990 2024-04-26T21:12:15.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.352+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f5de40ba110 con 0x7f5dec10a990 2024-04-26T21:12:15.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.410+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5dec062e10 con 0x7f5dd00a3990 2024-04-26T21:12:15.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: [26/Apr/2024:21:12:13] ENGINE Bus STARTING 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: [26/Apr/2024:21:12:13] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: [26/Apr/2024:21:12:13] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: [26/Apr/2024:21:12:13] ENGINE Bus STARTED 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:15 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.528+0000 7f5de2a00640 1 -- 172.21.15.27:0/2575753241 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5dec062e10 con 0x7f5dd00a3990 2024-04-26T21:12:15.529 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 msgr2=0x7f5dd00a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 0x7f5dd00a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5dec109e30 tx=0x7f5ddc005f90 comp rx=0 tx=0).stop 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 msgr2=0x7f5dec1a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005e30 tx=0x7f5de4003970 comp rx=0 tx=0).stop 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 shutdown_connections 2024-04-26T21:12:15.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dd00a3990 0x7f5dd00a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec109fd0 0x7f5dec19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5dec10a990 0x7f5dec1a6490 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 --2- 172.21.15.27:0/2575753241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec108de0 0x7f5dec19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 >> 172.21.15.27:0/2575753241 conn(0x7f5dec077dc0 msgr2=0x7f5dec10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 shutdown_connections 2024-04-26T21:12:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.531+0000 7f5deb200640 1 -- 172.21.15.27:0/2575753241 wait complete. 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: [26/Apr/2024:21:12:13] ENGINE Bus STARTING 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: [26/Apr/2024:21:12:13] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: [26/Apr/2024:21:12:13] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: [26/Apr/2024:21:12:13] ENGINE Bus STARTED 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:12:15.549 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:12:15.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:15 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: [26/Apr/2024:21:12:13] ENGINE Bus STARTING 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: [26/Apr/2024:21:12:13] ENGINE Serving on http://172.21.15.27:8765 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: [26/Apr/2024:21:12:13] ENGINE Serving on https://172.21.15.27:7150 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: [26/Apr/2024:21:12:13] ENGINE Bus STARTED 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2409661338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:15 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:15.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 -- 172.21.15.27:0/2646898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894104bc0 msgr2=0x7f2894104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 --2- 172.21.15.27:0/2646898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894104bc0 0x7f2894104fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2884009900 tx=0x7f288402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 -- 172.21.15.27:0/2646898048 shutdown_connections 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 --2- 172.21.15.27:0/2646898048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894105db0 0x7f2894106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 --2- 172.21.15.27:0/2646898048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894106770 0x7f289410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 --2- 172.21.15.27:0/2646898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894104bc0 0x7f2894104fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 -- 172.21.15.27:0/2646898048 >> 172.21.15.27:0/2646898048 conn(0x7f2894100150 msgr2=0x7f2894102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 -- 172.21.15.27:0/2646898048 shutdown_connections 2024-04-26T21:12:15.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.744+0000 7f289a200640 1 -- 172.21.15.27:0/2646898048 wait complete. 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.745+0000 7f289a200640 1 Processor -- start 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 -- start start 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 0x7f289419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 0x7f28941a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28941105e0 con 0x7f2894104bc0 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2894110460 con 0x7f2894105db0 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f289a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2894110760 con 0x7f2894106770 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f2893400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f2893400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32986/0 (socket says 172.21.15.27:32986) 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f2893400640 1 -- 172.21.15.27:0/2382909185 learned_addr learned my addr 172.21.15.27:0/2382909185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f2892a00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 0x7f289419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.746+0000 7f2893e00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 0x7f28941a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 -- 172.21.15.27:0/2382909185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 msgr2=0x7f28941a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 0x7f28941a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 -- 172.21.15.27:0/2382909185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 msgr2=0x7f289419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 0x7f289419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 -- 172.21.15.27:0/2382909185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28941a2890 con 0x7f2894104bc0 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2892a00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 0x7f289419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:15.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893e00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 0x7f28941a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.747+0000 7f2893400640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2884031460 tx=0x7f2884003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.748+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288403f070 con 0x7f2894104bc0 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.748+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941a2b20 con 0x7f2894104bc0 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.748+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2884031ba0 con 0x7f2894104bc0 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.748+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884035030 con 0x7f2894104bc0 2024-04-26T21:12:15.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.748+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a3010 con 0x7f2894104bc0 2024-04-26T21:12:15.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.749+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289410bee0 con 0x7f2894104bc0 2024-04-26T21:12:15.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.752+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f2884042070 con 0x7f2894104bc0 2024-04-26T21:12:15.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.754+0000 7f288be00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 0x7f28780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.754+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8549+0+0 (secure 0 0 0) 0x7f28840f1540 con 0x7f2894104bc0 2024-04-26T21:12:15.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.754+0000 7f2892a00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 0x7f28780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.755+0000 7f2892a00640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 0x7f28780a6280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2880005fd0 tx=0x7f288000b390 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:15.759+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f28840bdb60 con 0x7f2894104bc0 2024-04-26T21:12:16.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.017+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2894062d70 con 0x7f28780a3dc0 2024-04-26T21:12:16.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.030+0000 7f288be00640 1 -- 172.21.15.27:0/2382909185 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2894062d70 con 0x7f28780a3dc0 2024-04-26T21:12:16.031 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T21:12:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.032+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 msgr2=0x7f28780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.032+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 0x7f28780a6280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2880005fd0 tx=0x7f288000b390 comp rx=0 tx=0).stop 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 msgr2=0x7f289419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2884031460 tx=0x7f2884003c10 comp rx=0 tx=0).stop 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 shutdown_connections 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f28780a3dc0 0x7f28780a6280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894106770 0x7f28941a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2894105db0 0x7f289419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 --2- 172.21.15.27:0/2382909185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894104bc0 0x7f289419ac80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 >> 172.21.15.27:0/2382909185 conn(0x7f2894100150 msgr2=0x7f28941090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 shutdown_connections 2024-04-26T21:12:16.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.033+0000 7f289a200640 1 -- 172.21.15.27:0/2382909185 wait complete. 2024-04-26T21:12:16.053 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T21:12:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 -- 172.21.15.27:0/4148727705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 msgr2=0x7f78d010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 --2- 172.21.15.27:0/4148727705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d010d000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009ca0 tx=0x7f78c802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 -- 172.21.15.27:0/4148727705 shutdown_connections 2024-04-26T21:12:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 --2- 172.21.15.27:0/4148727705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0105db0 0x7f78d0106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 --2- 172.21.15.27:0/4148727705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d010d000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 --2- 172.21.15.27:0/4148727705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0104bc0 0x7f78d0104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 -- 172.21.15.27:0/4148727705 >> 172.21.15.27:0/4148727705 conn(0x7f78d0100150 msgr2=0x7f78d0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 -- 172.21.15.27:0/4148727705 shutdown_connections 2024-04-26T21:12:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.275+0000 7f78d5c00640 1 -- 172.21.15.27:0/4148727705 wait complete. 2024-04-26T21:12:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.276+0000 7f78d5c00640 1 Processor -- start 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.276+0000 7f78d5c00640 1 -- start start 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0105db0 0x7f78d019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d01a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01105d0 con 0x7f78d0106770 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0110450 con 0x7f78d0105db0 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78d5c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0110750 con 0x7f78d0104bc0 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d01a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78cea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d01a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33000/0 (socket says 172.21.15.27:33000) 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78cea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.27:53734/0 (socket says 172.21.15.27:53734) 2024-04-26T21:12:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78cf400640 1 -- 172.21.15.27:0/1664239923 learned_addr learned my addr 172.21.15.27:0/1664239923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.277+0000 7f78ce000640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0105db0 0x7f78d019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 -- 172.21.15.27:0/1664239923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0105db0 msgr2=0x7f78d019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0105db0 0x7f78d019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 -- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 msgr2=0x7f78d01a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d01a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 -- 172.21.15.27:0/1664239923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d01a2840 con 0x7f78d0104bc0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78cea00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78c000e9c0 tx=0x7f78c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c000ccc0 con 0x7f78d0104bc0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01a2b30 con 0x7f78d0104bc0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78c00044b0 con 0x7f78d0104bc0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.278+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0003e10 con 0x7f78d0104bc0 2024-04-26T21:12:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.279+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01a30d0 con 0x7f78d0104bc0 2024-04-26T21:12:16.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.280+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d010bee0 con 0x7f78d0104bc0 2024-04-26T21:12:16.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.282+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f78c0003fd0 con 0x7f78d0104bc0 2024-04-26T21:12:16.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.284+0000 7f78c7400640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 0x7f78b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.284+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8549+0+0 (secure 0 0 0) 0x7f78c00d0d40 con 0x7f78d0104bc0 2024-04-26T21:12:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.284+0000 7f78ce000640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 0x7f78b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.285+0000 7f78ce000640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 0x7f78b40a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009bd0 tx=0x7f78b8005f90 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.300+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f78c009d360 con 0x7f78d0104bc0 2024-04-26T21:12:16.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.562+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f78d0062d70 con 0x7f78b40a3e40 2024-04-26T21:12:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.574+0000 7f78c7400640 1 -- 172.21.15.27:0/1664239923 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f78d0062d70 con 0x7f78b40a3e40 2024-04-26T21:12:16.575 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 msgr2=0x7f78b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 0x7f78b40a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009bd0 tx=0x7f78b8005f90 comp rx=0 tx=0).stop 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 msgr2=0x7f78d019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78c000e9c0 tx=0x7f78c000ee80 comp rx=0 tx=0).stop 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 shutdown_connections 2024-04-26T21:12:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f78b40a3e40 0x7f78b40a6300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d0106770 0x7f78d01a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d0105db0 0x7f78d019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 --2- 172.21.15.27:0/1664239923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78d0104bc0 0x7f78d019ac30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 >> 172.21.15.27:0/1664239923 conn(0x7f78d0100150 msgr2=0x7f78d01090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 shutdown_connections 2024-04-26T21:12:16.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:12:16.577+0000 7f78d5c00640 1 -- 172.21.15.27:0/1664239923 wait complete. 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.594 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:16 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.597 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T21:12:16.676 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T21:12:16.686 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T21:12:16.720 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:12:16.721 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T21:12:16.721 INFO:tasks.workunit:timeout=3h 2024-04-26T21:12:16.721 INFO:tasks.workunit:cleanup=True 2024-04-26T21:12:16.721 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:16 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:16 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-26 21:09:48.629474097 +0000 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-26 21:09:48.629474097 +0000 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-26 21:12:12.737076289 +0000 2024-04-26T21:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-04-26T21:12:16.783 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:16.784 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T21:12:16.884 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:12:16.930 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T21:12:17.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:17 smithi027 ceph-mon[28085]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:17 smithi027 ceph-mon[28085]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:17 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:17.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:17 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:17 smithi204 ceph-mon[32145]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:17 smithi204 ceph-mon[32145]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:17 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:17 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:17.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:17 smithi094 ceph-mon[31273]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:17 smithi094 ceph-mon[31273]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:17 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:17 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.718 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:18.718 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.719 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:18 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:18 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:12:18.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:12:18.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-26T21:12:18.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.781 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:18.781 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.781 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.781 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:18.781 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:18 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:19.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:19 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:19 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:12:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:19 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:20.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:12:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:20 smithi027 ceph-mon[28085]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:12:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:20 smithi204 ceph-mon[32145]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:12:20.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:12:20.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 67 op/s 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-26T21:12:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:20 smithi094 ceph-mon[31273]: Updating smithi204:/etc/ceph/ceph.conf 2024-04-26T21:12:21.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:21 smithi027 ceph-mon[28085]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:21 smithi027 ceph-mon[28085]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:21 smithi027 ceph-mon[28085]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:21 smithi204 ceph-mon[32145]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:21 smithi204 ceph-mon[32145]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:21 smithi204 ceph-mon[32145]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:21 smithi094 ceph-mon[31273]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:21 smithi094 ceph-mon[31273]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:21.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:21 smithi094 ceph-mon[31273]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:22 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:22 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:22.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: Updating smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:12:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:22 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:23.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:23 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-04-26T21:12:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:23.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:23.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:23.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:23 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-04-26T21:12:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:23.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:23.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:23.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:23 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:12:26.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:25 smithi204 ceph-mon[32145]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-26T21:12:26.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:25 smithi094 ceph-mon[31273]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-26T21:12:26.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:25 smithi027 ceph-mon[28085]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-26T21:12:28.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:28 smithi094 ceph-mon[31273]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T21:12:28.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:28 smithi027 ceph-mon[28085]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T21:12:28.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:28 smithi204 ceph-mon[32145]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T21:12:30.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:30 smithi094 ceph-mon[31273]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-26T21:12:30.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:30 smithi027 ceph-mon[28085]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-26T21:12:30.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:30 smithi204 ceph-mon[32145]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-26T21:12:32.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:32 smithi094 ceph-mon[31273]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T21:12:32.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:32 smithi027 ceph-mon[28085]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T21:12:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:32 smithi204 ceph-mon[32145]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T21:12:34.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:34 smithi094 ceph-mon[31273]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T21:12:34.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:34 smithi027 ceph-mon[28085]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T21:12:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:34 smithi204 ceph-mon[32145]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-26T21:12:36.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:36 smithi027 ceph-mon[28085]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-26T21:12:36.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:36 smithi204 ceph-mon[32145]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-26T21:12:36.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:36 smithi094 ceph-mon[31273]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-26T21:12:38.348 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:38 smithi094 ceph-mon[31273]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:12:38.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:38 smithi027 ceph-mon[28085]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:12:38.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:38 smithi204 ceph-mon[32145]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:12:40.394 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:40 smithi027 ceph-mon[28085]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:12:40.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:40 smithi204 ceph-mon[32145]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:12:40.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:40 smithi094 ceph-mon[31273]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:12:42.305 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:42 smithi204 ceph-mon[32145]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-26T21:12:42.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:42 smithi027 ceph-mon[28085]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-26T21:12:42.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:42 smithi094 ceph-mon[31273]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-26T21:12:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:44 smithi027 ceph-mon[28085]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:12:44.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:44 smithi204 ceph-mon[32145]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:12:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:44 smithi094 ceph-mon[31273]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:12:46.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:46 smithi027 ceph-mon[28085]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:12:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:46 smithi204 ceph-mon[32145]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:12:46.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:46 smithi094 ceph-mon[31273]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:12:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:48 smithi027 ceph-mon[28085]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:48 smithi204 ceph-mon[32145]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:48.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:48 smithi094 ceph-mon[31273]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:50 smithi027 ceph-mon[28085]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:12:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:50 smithi204 ceph-mon[32145]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:12:50.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:50 smithi094 ceph-mon[31273]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:12:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:52 smithi027 ceph-mon[28085]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-26T21:12:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:52 smithi204 ceph-mon[32145]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-26T21:12:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:52 smithi094 ceph-mon[31273]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-26T21:12:54.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:54 smithi027 ceph-mon[28085]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:54.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:54 smithi204 ceph-mon[32145]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:54.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:54 smithi094 ceph-mon[31273]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:56.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:56 smithi027 ceph-mon[28085]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:12:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:56 smithi204 ceph-mon[32145]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:12:56.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:56 smithi094 ceph-mon[31273]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:12:58.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:12:58 smithi027 ceph-mon[28085]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:12:58 smithi204 ceph-mon[32145]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:12:58.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:12:58 smithi094 ceph-mon[31273]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-mon[28085]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:13:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:00 smithi204 ceph-mon[32145]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:13:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:00 smithi094 ceph-mon[31273]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:13:00.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:13:00.554+0000 7ff0e1a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:13:00.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:13:00.554+0000 7ff0e1a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:13:00.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:13:00.554+0000 7ff0e1a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:13:00.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:13:00.554+0000 7ff0e1a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:13:00.986 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:13:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:13:00.554+0000 7ff0e1a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:13:02.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:02 smithi027 ceph-mon[28085]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:13:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:02 smithi027 ceph-mon[28085]: mgrmap e24: x(active, since 48s), standbys: z, y 2024-04-26T21:13:02.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:02 smithi204 ceph-mon[32145]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:13:02.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:02 smithi204 ceph-mon[32145]: mgrmap e24: x(active, since 48s), standbys: z, y 2024-04-26T21:13:02.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:02 smithi094 ceph-mon[31273]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-26T21:13:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:02 smithi094 ceph-mon[31273]: mgrmap e24: x(active, since 48s), standbys: z, y 2024-04-26T21:13:04.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:04 smithi027 ceph-mon[28085]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:04 smithi204 ceph-mon[32145]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:04 smithi094 ceph-mon[31273]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:06.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:06 smithi027 ceph-mon[28085]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:06.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:06 smithi204 ceph-mon[32145]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:06.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:06 smithi094 ceph-mon[31273]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:08.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:08 smithi027 ceph-mon[28085]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:08.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:08 smithi204 ceph-mon[32145]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:08.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:08 smithi094 ceph-mon[31273]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:10 smithi027 ceph-mon[28085]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-26T21:13:10.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:10 smithi204 ceph-mon[32145]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-26T21:13:10.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:10 smithi094 ceph-mon[31273]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-26T21:13:12.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:12 smithi027 ceph-mon[28085]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-26T21:13:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:12 smithi204 ceph-mon[32145]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-26T21:13:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:12 smithi094 ceph-mon[31273]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-26T21:13:14.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:14 smithi027 ceph-mon[28085]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:14 smithi204 ceph-mon[32145]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:14.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:14 smithi094 ceph-mon[31273]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:13:16.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:16 smithi027 ceph-mon[28085]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:16.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:16 smithi204 ceph-mon[32145]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:16 smithi094 ceph-mon[31273]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:13:18.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:18 smithi027 ceph-mon[28085]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:13:18.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:18 smithi204 ceph-mon[32145]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:13:18.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:18 smithi094 ceph-mon[31273]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:13:20.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:20 smithi027 ceph-mon[28085]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-26T21:13:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:20 smithi204 ceph-mon[32145]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-26T21:13:20.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:20 smithi094 ceph-mon[31273]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-26T21:13:22.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:22 smithi027 ceph-mon[28085]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:13:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:22 smithi204 ceph-mon[32145]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:13:22.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:22 smithi094 ceph-mon[31273]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:13:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:24 smithi027 ceph-mon[28085]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:24 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:24.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:24 smithi204 ceph-mon[32145]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:24.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:24 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:24.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:24 smithi094 ceph-mon[31273]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:24.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:24 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:25.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:25 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:25.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:25 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:25.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:25 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:13:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:25 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:25 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:25.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:25 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:13:25.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:25 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:25 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:25.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:25 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:13:26.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:26 smithi027 ceph-mon[28085]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:26.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:26 smithi204 ceph-mon[32145]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:26.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:26 smithi094 ceph-mon[31273]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:28.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:28 smithi027 ceph-mon[28085]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:28.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:28 smithi204 ceph-mon[32145]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:28.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:28 smithi094 ceph-mon[31273]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:30.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:30 smithi027 ceph-mon[28085]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:30.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:30 smithi204 ceph-mon[32145]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:30.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:30 smithi094 ceph-mon[31273]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:32.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:32 smithi027 ceph-mon[28085]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:32 smithi204 ceph-mon[32145]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:32.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:32 smithi094 ceph-mon[31273]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:34.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:34 smithi027 ceph-mon[28085]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:34 smithi204 ceph-mon[32145]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:34.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:34 smithi094 ceph-mon[31273]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:36.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:36 smithi027 ceph-mon[28085]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:36.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:36 smithi204 ceph-mon[32145]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:36.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:36 smithi094 ceph-mon[31273]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:38.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:38 smithi027 ceph-mon[28085]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:38.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:38 smithi204 ceph-mon[32145]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:38.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:38 smithi094 ceph-mon[31273]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:40.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:40 smithi027 ceph-mon[28085]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:40.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:40 smithi204 ceph-mon[32145]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:40.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:40 smithi094 ceph-mon[31273]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:13:42.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:42 smithi027 ceph-mon[28085]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:42.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:42 smithi204 ceph-mon[32145]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:42.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:42 smithi094 ceph-mon[31273]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:44.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:44 smithi204 ceph-mon[32145]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:44 smithi094 ceph-mon[31273]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:44.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:44 smithi027 ceph-mon[28085]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:13:46.283 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 58% (7227/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 95% (11714/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T21:13:46.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:46 smithi094 ceph-mon[31273]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-26T21:13:46.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:46 smithi094 ceph-mon[31273]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-26T21:13:46.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:46 smithi027 ceph-mon[28085]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-26T21:13:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:46 smithi027 ceph-mon[28085]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-26T21:13:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:46 smithi204 ceph-mon[32145]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-26T21:13:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:46 smithi204 ceph-mon[32145]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-26T21:13:46.987 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-04-26T21:13:46.988 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.989 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-04-26T21:13:46.989 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.989 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T21:13:46.989 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-26T21:13:46.989 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T21:13:47.001 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T21:13:47.105 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T21:13:47.113 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:13:47.113 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T21:13:47.985 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T21:13:48.097 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T21:13:48.182 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:13:48.187 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:13:48.187 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T21:13:48.280 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:13:48.292 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:13:48.292 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T21:13:48.344 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-26T21:13:48.346 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-26T21:13:48.346 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-26T21:13:48.444 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T21:13:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:48 smithi094 ceph-mon[31273]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T21:13:48.551 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-04-26T21:13:48.551 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-04-26T21:13:48.551 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Fri Apr 26 21:13:48 2024 2024-04-26T21:13:48.552 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 1048576 kB 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 16 kB 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-04-26T21:13:48.553 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-04-26T21:13:48.554 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-26T21:13:48.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:48 smithi027 ceph-mon[28085]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T21:13:48.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:48 smithi204 ceph-mon[32145]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T21:13:50.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:50 smithi094 ceph-mon[31273]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-26T21:13:50.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:50 smithi027 ceph-mon[28085]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-26T21:13:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:50 smithi204 ceph-mon[32145]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-26T21:13:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:52 smithi027 ceph-mon[28085]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:52.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:52 smithi204 ceph-mon[32145]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:52.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:52 smithi094 ceph-mon[31273]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:54 smithi027 ceph-mon[28085]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:54 smithi204 ceph-mon[32145]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:54 smithi094 ceph-mon[31273]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-26T21:13:56.578 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:56 smithi094 ceph-mon[31273]: pgmap v54: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-04-26T21:13:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:56 smithi027 ceph-mon[28085]: pgmap v54: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-04-26T21:13:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:56 smithi204 ceph-mon[32145]: pgmap v54: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-04-26T21:13:58.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:13:58 smithi027 ceph-mon[28085]: pgmap v55: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 11 op/s 2024-04-26T21:13:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:13:58 smithi204 ceph-mon[32145]: pgmap v55: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 11 op/s 2024-04-26T21:13:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:13:58 smithi094 ceph-mon[31273]: pgmap v55: 97 pgs: 97 active+clean; 81 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.7 MiB/s wr, 11 op/s 2024-04-26T21:14:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:00 smithi027 ceph-mon[28085]: pgmap v56: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-04-26T21:14:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:00 smithi204 ceph-mon[32145]: pgmap v56: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-04-26T21:14:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:00 smithi094 ceph-mon[31273]: pgmap v56: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-04-26T21:14:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:02 smithi027 ceph-mon[28085]: pgmap v57: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:02 smithi204 ceph-mon[32145]: pgmap v57: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:02 smithi094 ceph-mon[31273]: pgmap v57: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:04.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:04 smithi027 ceph-mon[28085]: pgmap v58: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:04 smithi204 ceph-mon[32145]: pgmap v58: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:04 smithi094 ceph-mon[31273]: pgmap v58: 97 pgs: 97 active+clean; 165 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:06 smithi027 ceph-mon[28085]: pgmap v59: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 33 op/s 2024-04-26T21:14:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:06 smithi204 ceph-mon[32145]: pgmap v59: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 33 op/s 2024-04-26T21:14:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:06 smithi094 ceph-mon[31273]: pgmap v59: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 33 op/s 2024-04-26T21:14:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:08 smithi027 ceph-mon[28085]: pgmap v60: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:08 smithi204 ceph-mon[32145]: pgmap v60: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:08 smithi094 ceph-mon[31273]: pgmap v60: 97 pgs: 97 active+clean; 245 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:10 smithi027 ceph-mon[28085]: pgmap v61: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:10 smithi204 ceph-mon[32145]: pgmap v61: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:10.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:10 smithi094 ceph-mon[31273]: pgmap v61: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:12 smithi027 ceph-mon[28085]: pgmap v62: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:12 smithi204 ceph-mon[32145]: pgmap v62: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:12.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:12 smithi094 ceph-mon[31273]: pgmap v62: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:14 smithi027 ceph-mon[28085]: pgmap v63: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:14 smithi204 ceph-mon[32145]: pgmap v63: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:14 smithi094 ceph-mon[31273]: pgmap v63: 97 pgs: 97 active+clean; 329 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-04-26T21:14:16.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:16 smithi027 ceph-mon[28085]: pgmap v64: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:16 smithi204 ceph-mon[32145]: pgmap v64: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:16 smithi094 ceph-mon[31273]: pgmap v64: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-04-26T21:14:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:18 smithi027 ceph-mon[28085]: pgmap v65: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:18 smithi204 ceph-mon[32145]: pgmap v65: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:18 smithi094 ceph-mon[31273]: pgmap v65: 97 pgs: 97 active+clean; 417 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-04-26T21:14:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:20 smithi027 ceph-mon[28085]: pgmap v66: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:20 smithi204 ceph-mon[32145]: pgmap v66: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:20 smithi094 ceph-mon[31273]: pgmap v66: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:22 smithi027 ceph-mon[28085]: pgmap v67: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:22 smithi204 ceph-mon[32145]: pgmap v67: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:22 smithi094 ceph-mon[31273]: pgmap v67: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:24 smithi027 ceph-mon[28085]: pgmap v68: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:24 smithi204 ceph-mon[32145]: pgmap v68: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:24.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:24 smithi094 ceph-mon[31273]: pgmap v68: 97 pgs: 97 active+clean; 509 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:25.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:25 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:25.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:25 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:25 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:26.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:26 smithi094 ceph-mon[31273]: pgmap v69: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:26.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:26 smithi027 ceph-mon[28085]: pgmap v69: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:26 smithi204 ceph-mon[32145]: pgmap v69: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:27.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:27 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:27.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:27 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:27.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:27 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:14:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:27 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:27 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:27 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:14:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:27 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:27 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:27.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:27 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:14:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:28 smithi027 ceph-mon[28085]: pgmap v70: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:28 smithi204 ceph-mon[32145]: pgmap v70: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:28.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:28 smithi094 ceph-mon[31273]: pgmap v70: 97 pgs: 97 active+clean; 593 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:30 smithi027 ceph-mon[28085]: pgmap v71: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:30 smithi204 ceph-mon[32145]: pgmap v71: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:30.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:30 smithi094 ceph-mon[31273]: pgmap v71: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:31 smithi027 ceph-mon[28085]: pgmap v72: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:31.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:31 smithi204 ceph-mon[32145]: pgmap v72: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:31.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:31 smithi094 ceph-mon[31273]: pgmap v72: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:34 smithi027 ceph-mon[28085]: pgmap v73: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:34 smithi204 ceph-mon[32145]: pgmap v73: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:34 smithi094 ceph-mon[31273]: pgmap v73: 97 pgs: 97 active+clean; 705 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-04-26T21:14:36.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:36 smithi027 ceph-mon[28085]: pgmap v74: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:36.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:36 smithi204 ceph-mon[32145]: pgmap v74: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:36.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:36 smithi094 ceph-mon[31273]: pgmap v74: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:38.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:38 smithi027 ceph-mon[28085]: pgmap v75: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:14:38.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:38 smithi204 ceph-mon[32145]: pgmap v75: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:14:38.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:38 smithi094 ceph-mon[31273]: pgmap v75: 97 pgs: 97 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:14:40.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:40 smithi027 ceph-mon[28085]: pgmap v76: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:40.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:40 smithi204 ceph-mon[32145]: pgmap v76: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:40.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:40 smithi094 ceph-mon[31273]: pgmap v76: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 24 MiB/s wr, 18 op/s 2024-04-26T21:14:42.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:42 smithi027 ceph-mon[28085]: pgmap v77: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:42.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:42 smithi204 ceph-mon[32145]: pgmap v77: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:42.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:42 smithi094 ceph-mon[31273]: pgmap v77: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:44 smithi027 ceph-mon[28085]: pgmap v78: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:44.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:44 smithi204 ceph-mon[32145]: pgmap v78: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:44 smithi094 ceph-mon[31273]: pgmap v78: 97 pgs: 97 active+clean; 885 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:46 smithi027 ceph-mon[28085]: pgmap v79: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:46 smithi204 ceph-mon[32145]: pgmap v79: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:46.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:46 smithi094 ceph-mon[31273]: pgmap v79: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 17972.17 kB/sec 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 17971.70 kB/sec 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 17972.17 kB/sec 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 17972.17 kB/sec 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 17972.17 kB/sec 2024-04-26T21:14:46.905 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:14:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:48 smithi027 ceph-mon[28085]: pgmap v80: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:48 smithi204 ceph-mon[32145]: pgmap v80: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:48.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:48 smithi094 ceph-mon[31273]: pgmap v80: 97 pgs: 97 active+clean; 969 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:14:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:50 smithi027 ceph-mon[28085]: pgmap v81: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-04-26T21:14:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:50 smithi204 ceph-mon[32145]: pgmap v81: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-04-26T21:14:50.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:50 smithi094 ceph-mon[31273]: pgmap v81: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-04-26T21:14:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:52 smithi027 ceph-mon[28085]: pgmap v82: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:52 smithi204 ceph-mon[32145]: pgmap v82: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:52 smithi094 ceph-mon[31273]: pgmap v82: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:54.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:54 smithi027 ceph-mon[28085]: pgmap v83: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:54.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:54 smithi204 ceph-mon[32145]: pgmap v83: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:54.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:54 smithi094 ceph-mon[31273]: pgmap v83: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-04-26T21:14:56.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:56 smithi027 ceph-mon[28085]: pgmap v84: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-04-26T21:14:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:56 smithi204 ceph-mon[32145]: pgmap v84: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-04-26T21:14:56.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:56 smithi094 ceph-mon[31273]: pgmap v84: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-04-26T21:14:58.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:14:58 smithi027 ceph-mon[28085]: pgmap v85: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-04-26T21:14:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:14:58 smithi204 ceph-mon[32145]: pgmap v85: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-04-26T21:14:58.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:14:58 smithi094 ceph-mon[31273]: pgmap v85: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-04-26T21:15:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:00 smithi027 ceph-mon[28085]: pgmap v86: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-04-26T21:15:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:00 smithi204 ceph-mon[32145]: pgmap v86: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-04-26T21:15:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:00 smithi094 ceph-mon[31273]: pgmap v86: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-04-26T21:15:02.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:02 smithi027 ceph-mon[28085]: pgmap v87: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:02.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:02 smithi204 ceph-mon[32145]: pgmap v87: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:02 smithi094 ceph-mon[31273]: pgmap v87: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:04.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:04 smithi027 ceph-mon[28085]: pgmap v88: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:04 smithi204 ceph-mon[32145]: pgmap v88: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:04 smithi094 ceph-mon[31273]: pgmap v88: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-04-26T21:15:06.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:06 smithi027 ceph-mon[28085]: pgmap v89: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 33 op/s 2024-04-26T21:15:06.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:06 smithi204 ceph-mon[32145]: pgmap v89: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 33 op/s 2024-04-26T21:15:06.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:06 smithi094 ceph-mon[31273]: pgmap v89: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 33 op/s 2024-04-26T21:15:08.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:08 smithi027 ceph-mon[28085]: pgmap v90: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-26T21:15:08.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:08 smithi204 ceph-mon[32145]: pgmap v90: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-26T21:15:08.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:08 smithi094 ceph-mon[31273]: pgmap v90: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-26T21:15:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:10 smithi027 ceph-mon[28085]: pgmap v91: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:10.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:10 smithi204 ceph-mon[32145]: pgmap v91: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:10.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:10 smithi094 ceph-mon[31273]: pgmap v91: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:12.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:12 smithi027 ceph-mon[28085]: pgmap v92: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:12 smithi204 ceph-mon[32145]: pgmap v92: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:12 smithi094 ceph-mon[31273]: pgmap v92: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:14.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:14 smithi027 ceph-mon[28085]: pgmap v93: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:14 smithi204 ceph-mon[32145]: pgmap v93: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:14.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:14 smithi094 ceph-mon[31273]: pgmap v93: 97 pgs: 97 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 27 op/s 2024-04-26T21:15:16.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:16 smithi027 ceph-mon[28085]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:16.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:16 smithi204 ceph-mon[32145]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:16 smithi094 ceph-mon[31273]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-04-26T21:15:18.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:18 smithi027 ceph-mon[28085]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-04-26T21:15:18.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:18 smithi204 ceph-mon[32145]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-04-26T21:15:18.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:18 smithi094 ceph-mon[31273]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 18 MiB/s wr, 12 op/s 2024-04-26T21:15:20.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:20 smithi027 ceph-mon[28085]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-04-26T21:15:20.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:20 smithi204 ceph-mon[32145]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-04-26T21:15:20.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:20 smithi094 ceph-mon[31273]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-04-26T21:15:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:22 smithi027 ceph-mon[28085]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:22 smithi204 ceph-mon[32145]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:22.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:22 smithi094 ceph-mon[31273]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:24.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:24 smithi027 ceph-mon[28085]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:24.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:24 smithi204 ceph-mon[32145]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:24.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:24 smithi094 ceph-mon[31273]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:26.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:26 smithi027 ceph-mon[28085]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-04-26T21:15:26.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:26 smithi204 ceph-mon[32145]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-04-26T21:15:26.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:26 smithi094 ceph-mon[31273]: pgmap v99: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-04-26T21:15:28.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:28 smithi204 ceph-mon[32145]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:28.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:28 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:28.624 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:28 smithi094 ceph-mon[31273]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:28.624 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:28 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:28 smithi027 ceph-mon[28085]: pgmap v100: 97 pgs: 97 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:28 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:29 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:29 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:29.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:29 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:15:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:29 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:29 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:29.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:29 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:15:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:29 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:29 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:29.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:29 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:15:30.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:30 smithi204 ceph-mon[32145]: pgmap v101: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:30.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:30 smithi094 ceph-mon[31273]: pgmap v101: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:30 smithi027 ceph-mon[28085]: pgmap v101: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:32.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:32 smithi094 ceph-mon[31273]: pgmap v102: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:32 smithi027 ceph-mon[28085]: pgmap v102: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:32 smithi204 ceph-mon[32145]: pgmap v102: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:34 smithi094 ceph-mon[31273]: pgmap v103: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:34.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:34 smithi027 ceph-mon[28085]: pgmap v103: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:34 smithi204 ceph-mon[32145]: pgmap v103: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 12 op/s 2024-04-26T21:15:36.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:36 smithi094 ceph-mon[31273]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-04-26T21:15:36.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:36 smithi027 ceph-mon[28085]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-04-26T21:15:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:36 smithi204 ceph-mon[32145]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-04-26T21:15:37.887 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:37.887 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 21696.96 kB/sec 2024-04-26T21:15:37.888 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 21696.33 kB/sec 2024-04-26T21:15:37.888 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 21696.96 kB/sec 2024-04-26T21:15:37.888 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 21696.96 kB/sec 2024-04-26T21:15:37.888 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 21696.96 kB/sec 2024-04-26T21:15:37.888 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:15:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:38 smithi094 ceph-mon[31273]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 13 op/s 2024-04-26T21:15:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:38 smithi027 ceph-mon[28085]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 13 op/s 2024-04-26T21:15:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:38 smithi204 ceph-mon[32145]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 19 MiB/s wr, 13 op/s 2024-04-26T21:15:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:40 smithi027 ceph-mon[28085]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 4339790.00 kB/sec 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 4327913.51 kB/sec 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 4339790.00 kB/sec 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 4339790.00 kB/sec 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 4339790.00 kB/sec 2024-04-26T21:15:40.751 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:15:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:40 smithi204 ceph-mon[32145]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:40.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:40 smithi094 ceph-mon[31273]: pgmap v106: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-04-26T21:15:42.554 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:42 smithi204 ceph-mon[32145]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:42 smithi027 ceph-mon[28085]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:42.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:42 smithi094 ceph-mon[31273]: pgmap v107: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 3110559.00 kB/sec 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 3104840.63 kB/sec 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3110559.00 kB/sec 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3110559.00 kB/sec 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3110559.00 kB/sec 2024-04-26T21:15:43.182 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:15:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:44 smithi027 ceph-mon[28085]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:44 smithi204 ceph-mon[32145]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:44.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:44 smithi094 ceph-mon[31273]: pgmap v108: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 MiB/s wr, 12 op/s 2024-04-26T21:15:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:46 smithi027 ceph-mon[28085]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-04-26T21:15:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:46 smithi204 ceph-mon[32145]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-04-26T21:15:46.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:46 smithi094 ceph-mon[31273]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-04-26T21:15:48.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:48 smithi027 ceph-mon[28085]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-04-26T21:15:48.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:48 smithi204 ceph-mon[32145]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-04-26T21:15:48.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:48 smithi094 ceph-mon[31273]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-04-26T21:15:49.681 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.681 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.683 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.683 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-04-26T21:15:49.690 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T21:15:49.782 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T21:15:49.783 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Fri Apr 26 21:15:49 2024 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 1048576 kB 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 1024 kB 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-04-26T21:15:49.784 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-04-26T21:15:49.785 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-26T21:15:50.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:50 smithi027 ceph-mon[28085]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-04-26T21:15:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:50 smithi204 ceph-mon[32145]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-04-26T21:15:50.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:50 smithi094 ceph-mon[31273]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-04-26T21:15:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:52 smithi027 ceph-mon[28085]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:52.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:52 smithi204 ceph-mon[32145]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:52.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:52 smithi094 ceph-mon[31273]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:54 smithi027 ceph-mon[28085]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:54 smithi204 ceph-mon[32145]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:54 smithi094 ceph-mon[31273]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T21:15:56.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:56 smithi027 ceph-mon[28085]: pgmap v114: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 MiB/s wr, 48 op/s 2024-04-26T21:15:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:56 smithi204 ceph-mon[32145]: pgmap v114: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 MiB/s wr, 48 op/s 2024-04-26T21:15:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:56 smithi094 ceph-mon[31273]: pgmap v114: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 52 MiB/s wr, 48 op/s 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 145496.88 kB/sec 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 145449.76 kB/sec 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 145496.88 kB/sec 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 145496.88 kB/sec 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 145496.88 kB/sec 2024-04-26T21:15:57.030 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:15:58.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:15:58 smithi027 ceph-mon[28085]: pgmap v115: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 44 op/s 2024-04-26T21:15:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:15:58 smithi204 ceph-mon[32145]: pgmap v115: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 44 op/s 2024-04-26T21:15:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:15:58 smithi094 ceph-mon[31273]: pgmap v115: 97 pgs: 97 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 44 op/s 2024-04-26T21:16:00.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:00 smithi027 ceph-mon[28085]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-04-26T21:16:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:00 smithi204 ceph-mon[32145]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-04-26T21:16:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:00 smithi094 ceph-mon[31273]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 MiB/s wr, 57 op/s 2024-04-26T21:16:02.065 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:02.065 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 390316.12 kB/sec 2024-04-26T21:16:02.065 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 390133.83 kB/sec 2024-04-26T21:16:02.065 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 390316.12 kB/sec 2024-04-26T21:16:02.066 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 390316.12 kB/sec 2024-04-26T21:16:02.066 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 390316.12 kB/sec 2024-04-26T21:16:02.066 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:16:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:02 smithi027 ceph-mon[28085]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:02 smithi204 ceph-mon[32145]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:02 smithi094 ceph-mon[31273]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 5150023.00 kB/sec 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 5122251.98 kB/sec 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 5150023.00 kB/sec 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 5150023.00 kB/sec 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 5150023.00 kB/sec 2024-04-26T21:16:04.600 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:16:04.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:04 smithi027 ceph-mon[28085]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:04 smithi204 ceph-mon[32145]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:04 smithi094 ceph-mon[31273]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-04-26T21:16:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:06 smithi027 ceph-mon[28085]: pgmap v119: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-04-26T21:16:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:06 smithi204 ceph-mon[32145]: pgmap v119: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-04-26T21:16:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:06 smithi094 ceph-mon[31273]: pgmap v119: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 171 MiB/s wr, 94 op/s 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 3481384.75 kB/sec 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 3469577.77 kB/sec 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3481384.75 kB/sec 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3481384.75 kB/sec 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3481384.75 kB/sec 2024-04-26T21:16:06.997 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-04-26T21:16:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:08 smithi027 ceph-mon[28085]: pgmap v120: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 119 MiB/s wr, 53 op/s 2024-04-26T21:16:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:08 smithi204 ceph-mon[32145]: pgmap v120: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 119 MiB/s wr, 53 op/s 2024-04-26T21:16:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:08 smithi094 ceph-mon[31273]: pgmap v120: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 119 MiB/s wr, 53 op/s 2024-04-26T21:16:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:10 smithi027 ceph-mon[28085]: pgmap v121: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 119 MiB/s wr, 58 op/s 2024-04-26T21:16:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:10 smithi204 ceph-mon[32145]: pgmap v121: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 119 MiB/s wr, 58 op/s 2024-04-26T21:16:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:10 smithi094 ceph-mon[31273]: pgmap v121: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 119 MiB/s wr, 58 op/s 2024-04-26T21:16:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:12 smithi027 ceph-mon[28085]: pgmap v122: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:12 smithi204 ceph-mon[32145]: pgmap v122: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:12 smithi094 ceph-mon[31273]: pgmap v122: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:13.541 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.541 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.541 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.541 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-04-26T21:16:13.544 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T21:16:13.628 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-04-26T21:16:13.628 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-04-26T21:16:13.628 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Fri Apr 26 21:16:13 2024 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:16:13.629 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 10485760 kB 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 1024 kB 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-04-26T21:16:13.630 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-04-26T21:16:13.631 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-26T21:16:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:14 smithi027 ceph-mon[28085]: pgmap v123: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:14 smithi204 ceph-mon[32145]: pgmap v123: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:14 smithi094 ceph-mon[31273]: pgmap v123: 97 pgs: 97 active+clean; 2.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-04-26T21:16:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:16 smithi027 ceph-mon[28085]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 89 MiB/s wr, 73 op/s 2024-04-26T21:16:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:16 smithi204 ceph-mon[32145]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 89 MiB/s wr, 73 op/s 2024-04-26T21:16:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:16 smithi094 ceph-mon[31273]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 89 MiB/s wr, 73 op/s 2024-04-26T21:16:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:18 smithi027 ceph-mon[28085]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-26T21:16:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:18 smithi204 ceph-mon[32145]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-26T21:16:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:18 smithi094 ceph-mon[31273]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-26T21:16:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:20 smithi027 ceph-mon[28085]: pgmap v126: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-04-26T21:16:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:20 smithi204 ceph-mon[32145]: pgmap v126: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-04-26T21:16:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:20 smithi094 ceph-mon[31273]: pgmap v126: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 51 op/s 2024-04-26T21:16:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:22 smithi027 ceph-mon[28085]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:22 smithi204 ceph-mon[32145]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:22.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:22 smithi094 ceph-mon[31273]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:23.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:23 smithi027 ceph-mon[28085]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:23.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:23 smithi204 ceph-mon[32145]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:23.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:23 smithi094 ceph-mon[31273]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 59 MiB/s wr, 46 op/s 2024-04-26T21:16:26.479 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:26 smithi094 ceph-mon[31273]: pgmap v129: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 113 MiB/s wr, 65 op/s 2024-04-26T21:16:26.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:26 smithi027 ceph-mon[28085]: pgmap v129: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 113 MiB/s wr, 65 op/s 2024-04-26T21:16:26.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:26 smithi204 ceph-mon[32145]: pgmap v129: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 113 MiB/s wr, 65 op/s 2024-04-26T21:16:28.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:28 smithi027 ceph-mon[28085]: pgmap v130: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 110 MiB/s wr, 37 op/s 2024-04-26T21:16:28.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:28 smithi204 ceph-mon[32145]: pgmap v130: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 110 MiB/s wr, 37 op/s 2024-04-26T21:16:28.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:28 smithi094 ceph-mon[31273]: pgmap v130: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 110 MiB/s wr, 37 op/s 2024-04-26T21:16:30.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:30 smithi027 ceph-mon[28085]: pgmap v131: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 MiB/s wr, 55 op/s 2024-04-26T21:16:30.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:30 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:30.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:30 smithi204 ceph-mon[32145]: pgmap v131: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 MiB/s wr, 55 op/s 2024-04-26T21:16:30.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:30 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:30.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:30 smithi094 ceph-mon[31273]: pgmap v131: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 164 MiB/s wr, 55 op/s 2024-04-26T21:16:30.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:30 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:31.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:31 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:31.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:31 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:31.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:31 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:16:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:31 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:31 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:31.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:31 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:16:31.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:31 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:31.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:31 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:31.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:31 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:16:32.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:32 smithi027 ceph-mon[28085]: pgmap v132: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:32 smithi204 ceph-mon[32145]: pgmap v132: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:32.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:32 smithi094 ceph-mon[31273]: pgmap v132: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:34 smithi027 ceph-mon[28085]: pgmap v133: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:34 smithi204 ceph-mon[32145]: pgmap v133: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:34 smithi094 ceph-mon[31273]: pgmap v133: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 108 MiB/s wr, 36 op/s 2024-04-26T21:16:36.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:36 smithi027 ceph-mon[28085]: pgmap v134: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 180 MiB/s wr, 58 op/s 2024-04-26T21:16:36.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:36 smithi204 ceph-mon[32145]: pgmap v134: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 180 MiB/s wr, 58 op/s 2024-04-26T21:16:36.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:36 smithi094 ceph-mon[31273]: pgmap v134: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 180 MiB/s wr, 58 op/s 2024-04-26T21:16:38.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:38 smithi027 ceph-mon[28085]: pgmap v135: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 126 MiB/s wr, 40 op/s 2024-04-26T21:16:38.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:38 smithi204 ceph-mon[32145]: pgmap v135: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 126 MiB/s wr, 40 op/s 2024-04-26T21:16:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:38 smithi094 ceph-mon[31273]: pgmap v135: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 126 MiB/s wr, 40 op/s 2024-04-26T21:16:40.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:40 smithi027 ceph-mon[28085]: pgmap v136: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:40.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:40 smithi204 ceph-mon[32145]: pgmap v136: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:40.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:40 smithi094 ceph-mon[31273]: pgmap v136: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:42.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:42 smithi027 ceph-mon[28085]: pgmap v137: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:42.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:42 smithi204 ceph-mon[32145]: pgmap v137: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:42.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:42 smithi094 ceph-mon[31273]: pgmap v137: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:44.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:44 smithi027 ceph-mon[28085]: pgmap v138: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:44.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:44 smithi204 ceph-mon[32145]: pgmap v138: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:44.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:44 smithi094 ceph-mon[31273]: pgmap v138: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-04-26T21:16:46.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:46 smithi027 ceph-mon[28085]: pgmap v139: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:46.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:46 smithi204 ceph-mon[32145]: pgmap v139: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:46.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:46 smithi094 ceph-mon[31273]: pgmap v139: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 179 MiB/s wr, 57 op/s 2024-04-26T21:16:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:48 smithi027 ceph-mon[28085]: pgmap v140: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 107 MiB/s wr, 35 op/s 2024-04-26T21:16:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:48 smithi204 ceph-mon[32145]: pgmap v140: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 107 MiB/s wr, 35 op/s 2024-04-26T21:16:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:48 smithi094 ceph-mon[31273]: pgmap v140: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 107 MiB/s wr, 35 op/s 2024-04-26T21:16:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:50 smithi027 ceph-mon[28085]: pgmap v141: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 161 MiB/s wr, 53 op/s 2024-04-26T21:16:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:50 smithi204 ceph-mon[32145]: pgmap v141: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 161 MiB/s wr, 53 op/s 2024-04-26T21:16:50.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:50 smithi094 ceph-mon[31273]: pgmap v141: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 161 MiB/s wr, 53 op/s 2024-04-26T21:16:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:52 smithi027 ceph-mon[28085]: pgmap v142: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:52 smithi204 ceph-mon[32145]: pgmap v142: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:52 smithi094 ceph-mon[31273]: pgmap v142: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:54.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:54 smithi027 ceph-mon[28085]: pgmap v143: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:54.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:54 smithi204 ceph-mon[32145]: pgmap v143: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:54.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:54 smithi094 ceph-mon[31273]: pgmap v143: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:56 smithi204 ceph-mon[32145]: pgmap v144: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 162 MiB/s wr, 53 op/s 2024-04-26T21:16:56.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:56 smithi094 ceph-mon[31273]: pgmap v144: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 162 MiB/s wr, 53 op/s 2024-04-26T21:16:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:56 smithi027 ceph-mon[28085]: pgmap v144: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 162 MiB/s wr, 53 op/s 2024-04-26T21:16:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:16:58 smithi204 ceph-mon[32145]: pgmap v145: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:58.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:16:58 smithi094 ceph-mon[31273]: pgmap v145: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:16:58.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:16:58 smithi027 ceph-mon[28085]: pgmap v145: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:17:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:00 smithi094 ceph-mon[31273]: pgmap v146: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:00 smithi027 ceph-mon[28085]: pgmap v146: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:00 smithi204 ceph-mon[32145]: pgmap v146: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:02 smithi094 ceph-mon[31273]: pgmap v147: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:02 smithi027 ceph-mon[28085]: pgmap v147: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:02 smithi204 ceph-mon[32145]: pgmap v147: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:04 smithi027 ceph-mon[28085]: pgmap v148: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:04 smithi204 ceph-mon[32145]: pgmap v148: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:04.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:04 smithi094 ceph-mon[31273]: pgmap v148: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:06 smithi027 ceph-mon[28085]: pgmap v149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 MiB/s wr, 68 op/s 2024-04-26T21:17:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:06 smithi204 ceph-mon[32145]: pgmap v149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 MiB/s wr, 68 op/s 2024-04-26T21:17:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:06 smithi094 ceph-mon[31273]: pgmap v149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 163 MiB/s wr, 68 op/s 2024-04-26T21:17:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:08 smithi027 ceph-mon[28085]: pgmap v150: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:08 smithi204 ceph-mon[32145]: pgmap v150: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:08 smithi094 ceph-mon[31273]: pgmap v150: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:10 smithi027 ceph-mon[28085]: pgmap v151: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:10 smithi204 ceph-mon[32145]: pgmap v151: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:10.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:10 smithi094 ceph-mon[31273]: pgmap v151: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:12 smithi027 ceph-mon[28085]: pgmap v152: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:12 smithi204 ceph-mon[32145]: pgmap v152: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:12 smithi094 ceph-mon[31273]: pgmap v152: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:14 smithi027 ceph-mon[28085]: pgmap v153: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:14 smithi204 ceph-mon[32145]: pgmap v153: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:14.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:14 smithi094 ceph-mon[31273]: pgmap v153: 97 pgs: 97 active+clean; 9.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 109 MiB/s wr, 50 op/s 2024-04-26T21:17:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:16 smithi027 ceph-mon[28085]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:16 smithi204 ceph-mon[32145]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:16 smithi094 ceph-mon[31273]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 164 MiB/s wr, 68 op/s 2024-04-26T21:17:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:18 smithi027 ceph-mon[28085]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 110 MiB/s wr, 35 op/s 2024-04-26T21:17:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:18 smithi204 ceph-mon[32145]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 110 MiB/s wr, 35 op/s 2024-04-26T21:17:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:18 smithi094 ceph-mon[31273]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 110 MiB/s wr, 35 op/s 2024-04-26T21:17:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:20 smithi027 ceph-mon[28085]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:20 smithi204 ceph-mon[32145]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:20 smithi094 ceph-mon[31273]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:22 smithi027 ceph-mon[28085]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:22 smithi204 ceph-mon[32145]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:22.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:22 smithi094 ceph-mon[31273]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:24 smithi027 ceph-mon[28085]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:24 smithi204 ceph-mon[32145]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:24 smithi094 ceph-mon[31273]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:26.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:26 smithi027 ceph-mon[28085]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:26 smithi204 ceph-mon[32145]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:26.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:26 smithi094 ceph-mon[31273]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 163 MiB/s wr, 53 op/s 2024-04-26T21:17:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:28 smithi027 ceph-mon[28085]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:17:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:28 smithi204 ceph-mon[32145]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:17:28.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:28 smithi094 ceph-mon[31273]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 108 MiB/s wr, 35 op/s 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 136360.45 kB/sec 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 136357.21 kB/sec 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 136360.45 kB/sec 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 136360.45 kB/sec 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 136360.45 kB/sec 2024-04-26T21:17:30.566 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-04-26T21:17:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:30 smithi027 ceph-mon[28085]: pgmap v161: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:30 smithi204 ceph-mon[32145]: pgmap v161: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:30.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:30 smithi094 ceph-mon[31273]: pgmap v161: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 164 MiB/s wr, 53 op/s 2024-04-26T21:17:31.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:31 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:31.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:31 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:31.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:31 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:32.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:32 smithi027 ceph-mon[28085]: pgmap v162: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:32 smithi204 ceph-mon[32145]: pgmap v162: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:32.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:32 smithi094 ceph-mon[31273]: pgmap v162: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:33.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:33 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:33 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:33 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:17:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:33 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:33 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:33 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:17:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:33 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:33 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:33 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:17:34.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:34 smithi027 ceph-mon[28085]: pgmap v163: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:34 smithi204 ceph-mon[32145]: pgmap v163: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:34.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:34 smithi094 ceph-mon[31273]: pgmap v163: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 109 MiB/s wr, 35 op/s 2024-04-26T21:17:36.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:36 smithi027 ceph-mon[28085]: pgmap v164: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 140 MiB/s wr, 47 op/s 2024-04-26T21:17:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:36 smithi204 ceph-mon[32145]: pgmap v164: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 140 MiB/s wr, 47 op/s 2024-04-26T21:17:36.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:36 smithi094 ceph-mon[31273]: pgmap v164: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 140 MiB/s wr, 47 op/s 2024-04-26T21:17:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:38 smithi027 ceph-mon[28085]: pgmap v165: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 86 MiB/s wr, 29 op/s 2024-04-26T21:17:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:38 smithi204 ceph-mon[32145]: pgmap v165: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 86 MiB/s wr, 29 op/s 2024-04-26T21:17:38.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:38 smithi094 ceph-mon[31273]: pgmap v165: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 86 MiB/s wr, 29 op/s 2024-04-26T21:17:40.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:40 smithi027 ceph-mon[28085]: pgmap v166: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 264 MiB/s wr, 78 op/s 2024-04-26T21:17:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:40 smithi204 ceph-mon[32145]: pgmap v166: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 264 MiB/s wr, 78 op/s 2024-04-26T21:17:40.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:40 smithi094 ceph-mon[31273]: pgmap v166: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 264 MiB/s wr, 78 op/s 2024-04-26T21:17:42.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:42 smithi027 ceph-mon[28085]: pgmap v167: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:42.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:42 smithi204 ceph-mon[32145]: pgmap v167: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:42.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:42 smithi094 ceph-mon[31273]: pgmap v167: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:44.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:44 smithi027 ceph-mon[28085]: pgmap v168: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:44 smithi204 ceph-mon[32145]: pgmap v168: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:44.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:44 smithi094 ceph-mon[31273]: pgmap v168: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 209 MiB/s wr, 60 op/s 2024-04-26T21:17:45.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:45 smithi027 ceph-mon[28085]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 387 MiB/s wr, 109 op/s 2024-04-26T21:17:45.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:45 smithi204 ceph-mon[32145]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 387 MiB/s wr, 109 op/s 2024-04-26T21:17:45.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:45 smithi094 ceph-mon[31273]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 387 MiB/s wr, 109 op/s 2024-04-26T21:17:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:48 smithi027 ceph-mon[28085]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 357 MiB/s wr, 97 op/s 2024-04-26T21:17:48.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:48 smithi204 ceph-mon[32145]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 357 MiB/s wr, 97 op/s 2024-04-26T21:17:48.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:48 smithi094 ceph-mon[31273]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 357 MiB/s wr, 97 op/s 2024-04-26T21:17:50.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:50 smithi027 ceph-mon[28085]: pgmap v171: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 532 MiB/s wr, 145 op/s 2024-04-26T21:17:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:50 smithi204 ceph-mon[32145]: pgmap v171: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 532 MiB/s wr, 145 op/s 2024-04-26T21:17:50.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:50 smithi094 ceph-mon[31273]: pgmap v171: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 532 MiB/s wr, 145 op/s 2024-04-26T21:17:52.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:52 smithi027 ceph-mon[28085]: pgmap v172: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:52 smithi204 ceph-mon[32145]: pgmap v172: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:52 smithi094 ceph-mon[31273]: pgmap v172: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:54.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:54 smithi027 ceph-mon[28085]: pgmap v173: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:54.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:54 smithi204 ceph-mon[32145]: pgmap v173: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:54.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:54 smithi094 ceph-mon[31273]: pgmap v173: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T21:17:56.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:56 smithi027 ceph-mon[28085]: pgmap v174: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 526 MiB/s wr, 143 op/s 2024-04-26T21:17:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:56 smithi204 ceph-mon[32145]: pgmap v174: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 526 MiB/s wr, 143 op/s 2024-04-26T21:17:56.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:56 smithi094 ceph-mon[31273]: pgmap v174: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 526 MiB/s wr, 143 op/s 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 430403.50 kB/sec 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 430380.91 kB/sec 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 430403.50 kB/sec 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 430403.50 kB/sec 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 430403.50 kB/sec 2024-04-26T21:17:57.843 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-04-26T21:17:58.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:17:58 smithi027 ceph-mon[28085]: pgmap v175: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 348 MiB/s wr, 95 op/s 2024-04-26T21:17:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:17:58 smithi204 ceph-mon[32145]: pgmap v175: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 348 MiB/s wr, 95 op/s 2024-04-26T21:17:58.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:17:58 smithi094 ceph-mon[31273]: pgmap v175: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 348 MiB/s wr, 95 op/s 2024-04-26T21:18:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:00 smithi027 ceph-mon[28085]: pgmap v176: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 131 op/s 2024-04-26T21:18:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:00 smithi204 ceph-mon[32145]: pgmap v176: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 131 op/s 2024-04-26T21:18:00.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:00 smithi094 ceph-mon[31273]: pgmap v176: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 131 op/s 2024-04-26T21:18:02.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:02 smithi027 ceph-mon[28085]: pgmap v177: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:02.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:02 smithi204 ceph-mon[32145]: pgmap v177: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:02.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:02 smithi094 ceph-mon[31273]: pgmap v177: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:03.049 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 5956606.00 kB/sec 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 5953832.47 kB/sec 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 5956606.00 kB/sec 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 5956606.00 kB/sec 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 5956606.00 kB/sec 2024-04-26T21:18:03.050 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-04-26T21:18:04.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:04 smithi027 ceph-mon[28085]: pgmap v178: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:04 smithi204 ceph-mon[32145]: pgmap v178: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:04.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:04 smithi094 ceph-mon[31273]: pgmap v178: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 308 MiB/s wr, 85 op/s 2024-04-26T21:18:06.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:06 smithi027 ceph-mon[28085]: pgmap v179: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 308 MiB/s wr, 104 op/s 2024-04-26T21:18:06.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:06 smithi204 ceph-mon[32145]: pgmap v179: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 308 MiB/s wr, 104 op/s 2024-04-26T21:18:06.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:06 smithi094 ceph-mon[31273]: pgmap v179: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 308 MiB/s wr, 104 op/s 2024-04-26T21:18:07.813 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:18:07.813 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 3959707.25 kB/sec 2024-04-26T21:18:07.814 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 3958279.73 kB/sec 2024-04-26T21:18:07.814 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3959707.25 kB/sec 2024-04-26T21:18:07.814 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3959707.25 kB/sec 2024-04-26T21:18:07.814 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3959707.25 kB/sec 2024-04-26T21:18:07.814 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-04-26T21:18:08.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:08 smithi027 ceph-mon[28085]: pgmap v180: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 135 MiB/s wr, 57 op/s 2024-04-26T21:18:08.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:08 smithi204 ceph-mon[32145]: pgmap v180: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 135 MiB/s wr, 57 op/s 2024-04-26T21:18:08.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:08 smithi094 ceph-mon[31273]: pgmap v180: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 135 MiB/s wr, 57 op/s 2024-04-26T21:18:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:10 smithi027 ceph-mon[28085]: pgmap v181: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 135 MiB/s wr, 61 op/s 2024-04-26T21:18:10.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:10 smithi204 ceph-mon[32145]: pgmap v181: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 135 MiB/s wr, 61 op/s 2024-04-26T21:18:10.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:10 smithi094 ceph-mon[31273]: pgmap v181: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 135 MiB/s wr, 61 op/s 2024-04-26T21:18:12.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:12 smithi027 ceph-mon[28085]: pgmap v182: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 25 op/s 2024-04-26T21:18:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:12 smithi204 ceph-mon[32145]: pgmap v182: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 25 op/s 2024-04-26T21:18:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:12 smithi094 ceph-mon[31273]: pgmap v182: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 25 op/s 2024-04-26T21:18:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:14 smithi204 ceph-mon[32145]: pgmap v183: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:18:14.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:14 smithi094 ceph-mon[31273]: pgmap v183: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:18:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:14 smithi027 ceph-mon[28085]: pgmap v183: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T21:18:15.920 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:18:15.920 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:18:15.921 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-04-26T21:18:15.921 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-04-26T21:18:15.927 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T21:18:15.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:18:16.065 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-26T21:18:16.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T21:18:16.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:16 smithi027 ceph-mon[28085]: pgmap v184: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:18:16.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:16 smithi204 ceph-mon[32145]: pgmap v184: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:18:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:16 smithi094 ceph-mon[31273]: pgmap v184: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T21:18:17.165 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:17.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:18:17.283 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:18:17.283 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T21:18:17.294 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T21:18:17.294 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:18:17.294 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:18:17.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T21:18:17.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T21:18:18.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:18 smithi094 ceph-mon[31273]: pgmap v185: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:18:18.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:18 smithi027 ceph-mon[28085]: pgmap v185: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:18:18.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:18 smithi204 ceph-mon[32145]: pgmap v185: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:18:20.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:20 smithi094 ceph-mon[31273]: pgmap v186: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 48 op/s 2024-04-26T21:18:20.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:20 smithi027 ceph-mon[28085]: pgmap v186: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 48 op/s 2024-04-26T21:18:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:20 smithi204 ceph-mon[32145]: pgmap v186: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 48 op/s 2024-04-26T21:18:22.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:22 smithi094 ceph-mon[31273]: pgmap v187: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:22 smithi027 ceph-mon[28085]: pgmap v187: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:22 smithi204 ceph-mon[32145]: pgmap v187: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:24.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:24 smithi094 ceph-mon[31273]: pgmap v188: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:24 smithi027 ceph-mon[28085]: pgmap v188: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:24 smithi204 ceph-mon[32145]: pgmap v188: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 46 op/s 2024-04-26T21:18:26.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:26 smithi027 ceph-mon[28085]: pgmap v189: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:26 smithi204 ceph-mon[32145]: pgmap v189: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:26.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:26 smithi094 ceph-mon[31273]: pgmap v189: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:28.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:28 smithi027 ceph-mon[28085]: pgmap v190: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 223 op/s 2024-04-26T21:18:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:28 smithi204 ceph-mon[32145]: pgmap v190: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 223 op/s 2024-04-26T21:18:28.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:28 smithi094 ceph-mon[31273]: pgmap v190: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 223 op/s 2024-04-26T21:18:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:30 smithi027 ceph-mon[28085]: pgmap v191: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:30 smithi204 ceph-mon[32145]: pgmap v191: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:30.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:30 smithi094 ceph-mon[31273]: pgmap v191: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 227 op/s 2024-04-26T21:18:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:32 smithi027 ceph-mon[28085]: pgmap v192: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 188 op/s 2024-04-26T21:18:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:32 smithi204 ceph-mon[32145]: pgmap v192: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 188 op/s 2024-04-26T21:18:32.827 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:32 smithi094 ceph-mon[31273]: pgmap v192: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 188 op/s 2024-04-26T21:18:33.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:33 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:33.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:33 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:33.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:33 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:34.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:34 smithi027 ceph-mon[28085]: pgmap v193: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 185 op/s 2024-04-26T21:18:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:34 smithi204 ceph-mon[32145]: pgmap v193: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 185 op/s 2024-04-26T21:18:34.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:34 smithi094 ceph-mon[31273]: pgmap v193: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 185 op/s 2024-04-26T21:18:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:35 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:35 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:35.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:35 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:18:35.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:35 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:35.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:35 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:35.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:35 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:18:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:35 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:35 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:35.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:35 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:18:36.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:36 smithi027 ceph-mon[28085]: pgmap v194: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 189 op/s 2024-04-26T21:18:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:36 smithi204 ceph-mon[32145]: pgmap v194: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 189 op/s 2024-04-26T21:18:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:36 smithi094 ceph-mon[31273]: pgmap v194: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 189 op/s 2024-04-26T21:18:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:38 smithi027 ceph-mon[28085]: pgmap v195: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:38 smithi204 ceph-mon[32145]: pgmap v195: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:38.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:38 smithi094 ceph-mon[31273]: pgmap v195: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:40 smithi027 ceph-mon[28085]: pgmap v196: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:40 smithi204 ceph-mon[32145]: pgmap v196: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:40 smithi094 ceph-mon[31273]: pgmap v196: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:42 smithi027 ceph-mon[28085]: pgmap v197: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:42.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:42 smithi204 ceph-mon[32145]: pgmap v197: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:42.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:42 smithi094 ceph-mon[31273]: pgmap v197: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:44.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:44 smithi027 ceph-mon[28085]: pgmap v198: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:44 smithi204 ceph-mon[32145]: pgmap v198: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:44.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:44 smithi094 ceph-mon[31273]: pgmap v198: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:46 smithi027 ceph-mon[28085]: pgmap v199: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:46 smithi204 ceph-mon[32145]: pgmap v199: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:46.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:46 smithi094 ceph-mon[31273]: pgmap v199: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:48.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:48 smithi027 ceph-mon[28085]: pgmap v200: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:48.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:48 smithi204 ceph-mon[32145]: pgmap v200: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:48.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:48 smithi094 ceph-mon[31273]: pgmap v200: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:50.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:50 smithi027 ceph-mon[28085]: pgmap v201: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:50 smithi204 ceph-mon[32145]: pgmap v201: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:50.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:50 smithi094 ceph-mon[31273]: pgmap v201: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:52 smithi027 ceph-mon[28085]: pgmap v202: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:52.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:52 smithi204 ceph-mon[32145]: pgmap v202: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:52.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:52 smithi094 ceph-mon[31273]: pgmap v202: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:54 smithi027 ceph-mon[28085]: pgmap v203: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:54 smithi204 ceph-mon[32145]: pgmap v203: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:54 smithi094 ceph-mon[31273]: pgmap v203: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:56 smithi027 ceph-mon[28085]: pgmap v204: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:56.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:56 smithi204 ceph-mon[32145]: pgmap v204: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:56 smithi094 ceph-mon[31273]: pgmap v204: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:18:58.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:18:58 smithi027 ceph-mon[28085]: pgmap v205: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:18:58 smithi204 ceph-mon[32145]: pgmap v205: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:18:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:18:58 smithi094 ceph-mon[31273]: pgmap v205: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:00 smithi027 ceph-mon[28085]: pgmap v206: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:00 smithi204 ceph-mon[32145]: pgmap v206: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:00 smithi094 ceph-mon[31273]: pgmap v206: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:02 smithi027 ceph-mon[28085]: pgmap v207: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:02 smithi204 ceph-mon[32145]: pgmap v207: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:02.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:02 smithi094 ceph-mon[31273]: pgmap v207: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:04 smithi027 ceph-mon[28085]: pgmap v208: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:04 smithi204 ceph-mon[32145]: pgmap v208: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:04.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:04 smithi094 ceph-mon[31273]: pgmap v208: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:06 smithi027 ceph-mon[28085]: pgmap v209: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:06 smithi204 ceph-mon[32145]: pgmap v209: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:06 smithi094 ceph-mon[31273]: pgmap v209: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:08 smithi027 ceph-mon[28085]: pgmap v210: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:08 smithi204 ceph-mon[32145]: pgmap v210: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:08 smithi094 ceph-mon[31273]: pgmap v210: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:10 smithi027 ceph-mon[28085]: pgmap v211: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:10 smithi204 ceph-mon[32145]: pgmap v211: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:10 smithi094 ceph-mon[31273]: pgmap v211: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:19:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:12 smithi027 ceph-mon[28085]: pgmap v212: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:12 smithi027 ceph-mon[28085]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:19:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:12 smithi204 ceph-mon[32145]: pgmap v212: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:12 smithi204 ceph-mon[32145]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:19:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:12 smithi094 ceph-mon[31273]: pgmap v212: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:19:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:12 smithi094 ceph-mon[31273]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:19:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:14 smithi027 ceph-mon[28085]: pgmap v214: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:19:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:14 smithi204 ceph-mon[32145]: pgmap v214: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:19:14.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:14 smithi094 ceph-mon[31273]: pgmap v214: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:19:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:16 smithi027 ceph-mon[28085]: pgmap v215: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:16 smithi204 ceph-mon[32145]: pgmap v215: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:16 smithi094 ceph-mon[31273]: pgmap v215: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:18 smithi027 ceph-mon[28085]: pgmap v216: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:18 smithi204 ceph-mon[32145]: pgmap v216: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:18 smithi094 ceph-mon[31273]: pgmap v216: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:19.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:19 smithi027 ceph-mon[28085]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:19:19.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:19 smithi027 ceph-mon[28085]: pgmap v218: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-26T21:19:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:19 smithi204 ceph-mon[32145]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:19:20.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:19 smithi204 ceph-mon[32145]: pgmap v218: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-26T21:19:20.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:19 smithi094 ceph-mon[31273]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:19:20.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:19 smithi094 ceph-mon[31273]: pgmap v218: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-26T21:19:22.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:22 smithi027 ceph-mon[28085]: pgmap v219: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 317 B/s wr, 10 op/s 2024-04-26T21:19:22.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:22 smithi204 ceph-mon[32145]: pgmap v219: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 317 B/s wr, 10 op/s 2024-04-26T21:19:22.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:22 smithi094 ceph-mon[31273]: pgmap v219: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 317 B/s wr, 10 op/s 2024-04-26T21:19:24.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:24 smithi027 ceph-mon[28085]: pgmap v220: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-26T21:19:24.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:24 smithi204 ceph-mon[32145]: pgmap v220: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-26T21:19:24.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:24 smithi094 ceph-mon[31273]: pgmap v220: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-26T21:19:26.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:26 smithi027 ceph-mon[28085]: pgmap v221: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:26.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:26 smithi204 ceph-mon[32145]: pgmap v221: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:26.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:26 smithi094 ceph-mon[31273]: pgmap v221: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:28.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:28 smithi027 ceph-mon[28085]: pgmap v222: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:28.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:28 smithi204 ceph-mon[32145]: pgmap v222: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:28.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:28 smithi094 ceph-mon[31273]: pgmap v222: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:19:29.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:29 smithi027 ceph-mon[28085]: pgmap v223: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 196 B/s wr, 14 op/s 2024-04-26T21:19:30.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:29 smithi204 ceph-mon[32145]: pgmap v223: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 196 B/s wr, 14 op/s 2024-04-26T21:19:30.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:29 smithi094 ceph-mon[31273]: pgmap v223: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 196 B/s wr, 14 op/s 2024-04-26T21:19:32.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:32 smithi027 ceph-mon[28085]: pgmap v224: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:32.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:32 smithi204 ceph-mon[32145]: pgmap v224: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:32.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:32 smithi094 ceph-mon[31273]: pgmap v224: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:34 smithi027 ceph-mon[28085]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:34.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:34 smithi204 ceph-mon[32145]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:34.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:34 smithi094 ceph-mon[31273]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:35.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:35 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:35.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:35 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:35.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:35 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:36.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:36 smithi027 ceph-mon[28085]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:36.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:36 smithi204 ceph-mon[32145]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:36.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:36 smithi094 ceph-mon[31273]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:37 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:37 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:37.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:37 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:19:37.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:37 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:37.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:37 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:37.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:37 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:19:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:37 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:37 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:37.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:37 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:19:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:38 smithi027 ceph-mon[28085]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:38 smithi204 ceph-mon[32145]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:38.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:38 smithi094 ceph-mon[31273]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:40 smithi027 ceph-mon[28085]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:40 smithi204 ceph-mon[32145]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:40 smithi094 ceph-mon[31273]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:42 smithi027 ceph-mon[28085]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:42.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:42 smithi204 ceph-mon[32145]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:42.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:42 smithi094 ceph-mon[31273]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:44 smithi027 ceph-mon[28085]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:44 smithi204 ceph-mon[32145]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:44.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:44 smithi094 ceph-mon[31273]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:45.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:45 smithi027 ceph-mon[28085]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:46.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:45 smithi204 ceph-mon[32145]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:46.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:45 smithi094 ceph-mon[31273]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:48.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:48 smithi027 ceph-mon[28085]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:48.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:48 smithi204 ceph-mon[32145]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:48.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:48 smithi094 ceph-mon[31273]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:50 smithi027 ceph-mon[28085]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:50 smithi204 ceph-mon[32145]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:50.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:50 smithi094 ceph-mon[31273]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:52.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:52 smithi027 ceph-mon[28085]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:52 smithi204 ceph-mon[32145]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:52 smithi094 ceph-mon[31273]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:54.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:54 smithi027 ceph-mon[28085]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:54.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:54 smithi204 ceph-mon[32145]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:54.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:54 smithi094 ceph-mon[31273]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:56.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:56 smithi027 ceph-mon[28085]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:56.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:56 smithi204 ceph-mon[32145]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:56.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:56 smithi094 ceph-mon[31273]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:19:58.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:19:58 smithi027 ceph-mon[28085]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:58.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:19:58 smithi204 ceph-mon[32145]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:19:58.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:19:58 smithi094 ceph-mon[31273]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:00 smithi027 ceph-mon[28085]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:00.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:00 smithi027 ceph-mon[28085]: overall HEALTH_OK 2024-04-26T21:20:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:00 smithi204 ceph-mon[32145]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:00 smithi204 ceph-mon[32145]: overall HEALTH_OK 2024-04-26T21:20:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:00 smithi094 ceph-mon[31273]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:00.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:00 smithi094 ceph-mon[31273]: overall HEALTH_OK 2024-04-26T21:20:02.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:02 smithi204 ceph-mon[32145]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:02.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:02 smithi094 ceph-mon[31273]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:02 smithi027 ceph-mon[28085]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:04.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:04 smithi204 ceph-mon[32145]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:04.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:04 smithi094 ceph-mon[31273]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:04 smithi027 ceph-mon[28085]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:06.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:06 smithi094 ceph-mon[31273]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:20:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:06 smithi027 ceph-mon[28085]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:20:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:06 smithi204 ceph-mon[32145]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:20:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:07 smithi027 ceph-mon[28085]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T21:20:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:07 smithi204 ceph-mon[32145]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T21:20:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:07 smithi094 ceph-mon[31273]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T21:20:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:08 smithi027 ceph-mon[28085]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:08 smithi204 ceph-mon[32145]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:08 smithi094 ceph-mon[31273]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:10 smithi027 ceph-mon[28085]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:10 smithi204 ceph-mon[32145]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:10.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:10 smithi094 ceph-mon[31273]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:12.555 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:12 smithi204 ceph-mon[32145]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:12 smithi027 ceph-mon[28085]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:12 smithi094 ceph-mon[31273]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:14 smithi027 ceph-mon[28085]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:14 smithi027 ceph-mon[28085]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T21:20:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:14 smithi204 ceph-mon[32145]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:14 smithi204 ceph-mon[32145]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T21:20:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:14 smithi094 ceph-mon[31273]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:20:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:14 smithi094 ceph-mon[31273]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T21:20:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:16 smithi027 ceph-mon[28085]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-26T21:20:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:16 smithi204 ceph-mon[32145]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-26T21:20:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:16 smithi094 ceph-mon[31273]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-26T21:20:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:18 smithi027 ceph-mon[28085]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:20:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:18 smithi204 ceph-mon[32145]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:20:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:18 smithi094 ceph-mon[31273]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:20:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:20 smithi027 ceph-mon[28085]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:20 smithi204 ceph-mon[32145]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:20.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:20 smithi094 ceph-mon[31273]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:22 smithi027 ceph-mon[28085]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:22 smithi204 ceph-mon[32145]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:22.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:22 smithi094 ceph-mon[31273]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:24 smithi027 ceph-mon[28085]: pgmap v252: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:24 smithi204 ceph-mon[32145]: pgmap v252: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:24 smithi094 ceph-mon[31273]: pgmap v252: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:20:26.603 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:26 smithi094 ceph-mon[31273]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:20:26.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:26 smithi027 ceph-mon[28085]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:20:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:26 smithi204 ceph-mon[32145]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:20:28.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:28 smithi094 ceph-mon[31273]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:28 smithi027 ceph-mon[28085]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:28 smithi204 ceph-mon[32145]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:30 smithi027 ceph-mon[28085]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:30 smithi204 ceph-mon[32145]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:30.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:30 smithi094 ceph-mon[31273]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:32 smithi027 ceph-mon[28085]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:32 smithi204 ceph-mon[32145]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:32.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:32 smithi094 ceph-mon[31273]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:34.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:34 smithi027 ceph-mon[28085]: pgmap v257: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:34 smithi204 ceph-mon[32145]: pgmap v257: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:34.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:34 smithi094 ceph-mon[31273]: pgmap v257: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:36.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:36 smithi027 ceph-mon[28085]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:36 smithi204 ceph-mon[32145]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:36 smithi094 ceph-mon[31273]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:37.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:37 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:37.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:37 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:37.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:37 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:38 smithi094 ceph-mon[31273]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:38 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:38 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:38 smithi027 ceph-mon[28085]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:38 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:38 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:38 smithi204 ceph-mon[32145]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:38 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:38 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:39.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:39 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:20:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:39 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:20:39.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:39 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:20:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:40 smithi027 ceph-mon[28085]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:40 smithi204 ceph-mon[32145]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:40 smithi094 ceph-mon[31273]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:42 smithi027 ceph-mon[28085]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:42.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:42 smithi204 ceph-mon[32145]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:42.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:42 smithi094 ceph-mon[31273]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:44 smithi027 ceph-mon[28085]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:44 smithi204 ceph-mon[32145]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:44.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:44 smithi094 ceph-mon[31273]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:46 smithi027 ceph-mon[28085]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:46 smithi204 ceph-mon[32145]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:46.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:46 smithi094 ceph-mon[31273]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:48.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:48 smithi027 ceph-mon[28085]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:48.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:48 smithi204 ceph-mon[32145]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:48.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:48 smithi094 ceph-mon[31273]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:50.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:50 smithi027 ceph-mon[28085]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:50 smithi204 ceph-mon[32145]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:50.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:50 smithi094 ceph-mon[31273]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:52 smithi027 ceph-mon[28085]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:52.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:52 smithi204 ceph-mon[32145]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:52.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:52 smithi094 ceph-mon[31273]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:54 smithi027 ceph-mon[28085]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:54 smithi204 ceph-mon[32145]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:54.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:54 smithi094 ceph-mon[31273]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:56 smithi027 ceph-mon[28085]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:56 smithi204 ceph-mon[32145]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:56.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:56 smithi094 ceph-mon[31273]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:20:58.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:20:58 smithi027 ceph-mon[28085]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:20:58 smithi204 ceph-mon[32145]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:20:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:20:58 smithi094 ceph-mon[31273]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:00 smithi027 ceph-mon[28085]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:00 smithi204 ceph-mon[32145]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:00 smithi094 ceph-mon[31273]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:02 smithi027 ceph-mon[28085]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:02 smithi204 ceph-mon[32145]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:02 smithi094 ceph-mon[31273]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:04 smithi027 ceph-mon[28085]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:04 smithi204 ceph-mon[32145]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:04 smithi094 ceph-mon[31273]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:06 smithi204 ceph-mon[32145]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:21:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:06 smithi094 ceph-mon[31273]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:21:06.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:06 smithi027 ceph-mon[28085]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:21:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:07 smithi094 ceph-mon[31273]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T21:21:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:07 smithi094 ceph-mon[31273]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:07 smithi027 ceph-mon[28085]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T21:21:07.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:07 smithi027 ceph-mon[28085]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:07 smithi204 ceph-mon[32145]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T21:21:08.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:07 smithi204 ceph-mon[32145]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:10 smithi027 ceph-mon[28085]: pgmap v276: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:10.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:10 smithi204 ceph-mon[32145]: pgmap v276: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:10.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:10 smithi094 ceph-mon[31273]: pgmap v276: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:12.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:12 smithi204 ceph-mon[32145]: pgmap v277: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:12.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:12 smithi094 ceph-mon[31273]: pgmap v277: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:12 smithi027 ceph-mon[28085]: pgmap v277: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:14 smithi204 ceph-mon[32145]: pgmap v278: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:14.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:14 smithi204 ceph-mon[32145]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T21:21:14.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:14 smithi094 ceph-mon[31273]: pgmap v278: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:14.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:14 smithi094 ceph-mon[31273]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T21:21:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:14 smithi027 ceph-mon[28085]: pgmap v278: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T21:21:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:14 smithi027 ceph-mon[28085]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T21:21:16.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:16 smithi094 ceph-mon[31273]: pgmap v280: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 11 op/s 2024-04-26T21:21:16.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:16 smithi027 ceph-mon[28085]: pgmap v280: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 11 op/s 2024-04-26T21:21:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:16 smithi204 ceph-mon[32145]: pgmap v280: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 11 op/s 2024-04-26T21:21:18.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:18 smithi094 ceph-mon[31273]: pgmap v281: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:21:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:18 smithi027 ceph-mon[28085]: pgmap v281: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:21:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:18 smithi204 ceph-mon[32145]: pgmap v281: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:21:20.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:20 smithi094 ceph-mon[31273]: pgmap v282: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:20.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:20 smithi027 ceph-mon[28085]: pgmap v282: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:20.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:20 smithi204 ceph-mon[32145]: pgmap v282: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:22.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:22 smithi094 ceph-mon[31273]: pgmap v283: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:22 smithi027 ceph-mon[28085]: pgmap v283: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:22.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:22 smithi204 ceph-mon[32145]: pgmap v283: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:24.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:24 smithi027 ceph-mon[28085]: pgmap v284: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:24 smithi204 ceph-mon[32145]: pgmap v284: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:24.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:24 smithi094 ceph-mon[31273]: pgmap v284: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T21:21:26.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:26 smithi094 ceph-mon[31273]: pgmap v285: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:21:26.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:26 smithi027 ceph-mon[28085]: pgmap v285: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:21:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:26 smithi204 ceph-mon[32145]: pgmap v285: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 89 B/s wr, 13 op/s 2024-04-26T21:21:28.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:28 smithi027 ceph-mon[28085]: pgmap v286: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:28 smithi204 ceph-mon[32145]: pgmap v286: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:28.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:28 smithi094 ceph-mon[31273]: pgmap v286: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:30.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:30 smithi027 ceph-mon[28085]: pgmap v287: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:30 smithi204 ceph-mon[32145]: pgmap v287: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:30.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:30 smithi094 ceph-mon[31273]: pgmap v287: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:32 smithi027 ceph-mon[28085]: pgmap v288: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:32 smithi204 ceph-mon[32145]: pgmap v288: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:32.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:32 smithi094 ceph-mon[31273]: pgmap v288: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:34.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:34 smithi027 ceph-mon[28085]: pgmap v289: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:34 smithi204 ceph-mon[32145]: pgmap v289: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:34.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:34 smithi094 ceph-mon[31273]: pgmap v289: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:36.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:36 smithi027 ceph-mon[28085]: pgmap v290: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:36 smithi204 ceph-mon[32145]: pgmap v290: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:36 smithi094 ceph-mon[31273]: pgmap v290: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:38 smithi027 ceph-mon[28085]: pgmap v291: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:38 smithi204 ceph-mon[32145]: pgmap v291: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:38.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:38 smithi094 ceph-mon[31273]: pgmap v291: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:39.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:39 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:39.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:39 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:39.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:39 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:40 smithi027 ceph-mon[28085]: pgmap v292: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:40 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:40 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:40 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:40 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:40 smithi204 ceph-mon[32145]: pgmap v292: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:40 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:40 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:40 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:40 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:40 smithi094 ceph-mon[31273]: pgmap v292: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:40 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:40 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:40 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:40.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:40 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:41 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:41 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:41.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:41 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:41.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:41 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:41.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:41 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:41.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:41 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:41 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:41 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:41.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:41 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:21:42.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:42 smithi204 ceph-mon[32145]: pgmap v293: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:42 smithi027 ceph-mon[28085]: pgmap v293: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:42.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:42 smithi094 ceph-mon[31273]: pgmap v293: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:44.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:44 smithi027 ceph-mon[28085]: pgmap v294: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:44.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:44 smithi204 ceph-mon[32145]: pgmap v294: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:44.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:44 smithi094 ceph-mon[31273]: pgmap v294: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:46.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:46 smithi027 ceph-mon[28085]: pgmap v295: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:46.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:46 smithi204 ceph-mon[32145]: pgmap v295: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:46.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:46 smithi094 ceph-mon[31273]: pgmap v295: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:48.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:48 smithi027 ceph-mon[28085]: pgmap v296: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:48.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:48 smithi204 ceph-mon[32145]: pgmap v296: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:48.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:48 smithi094 ceph-mon[31273]: pgmap v296: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:50.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:50 smithi027 ceph-mon[28085]: pgmap v297: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:50.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:50 smithi204 ceph-mon[32145]: pgmap v297: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:50.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:50 smithi094 ceph-mon[31273]: pgmap v297: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:52 smithi027 ceph-mon[28085]: pgmap v298: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:52.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:52 smithi204 ceph-mon[32145]: pgmap v298: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:52.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:52 smithi094 ceph-mon[31273]: pgmap v298: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:54 smithi027 ceph-mon[28085]: pgmap v299: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:54.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:54 smithi204 ceph-mon[32145]: pgmap v299: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:54.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:54 smithi094 ceph-mon[31273]: pgmap v299: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:56 smithi027 ceph-mon[28085]: pgmap v300: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:56.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:56 smithi204 ceph-mon[32145]: pgmap v300: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:56.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:56 smithi094 ceph-mon[31273]: pgmap v300: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:21:58.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:21:58 smithi027 ceph-mon[28085]: pgmap v301: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:58.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:21:58 smithi204 ceph-mon[32145]: pgmap v301: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:21:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:21:58 smithi094 ceph-mon[31273]: pgmap v301: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:00 smithi027 ceph-mon[28085]: pgmap v302: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:00.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:00 smithi204 ceph-mon[32145]: pgmap v302: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:00 smithi094 ceph-mon[31273]: pgmap v302: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:02 smithi027 ceph-mon[28085]: pgmap v303: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:02.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:02 smithi204 ceph-mon[32145]: pgmap v303: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:02 smithi094 ceph-mon[31273]: pgmap v303: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:04 smithi027 ceph-mon[28085]: pgmap v304: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:04.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:04 smithi204 ceph-mon[32145]: pgmap v304: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:04 smithi094 ceph-mon[31273]: pgmap v304: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:06 smithi027 ceph-mon[28085]: pgmap v305: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:22:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:06 smithi204 ceph-mon[32145]: pgmap v305: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:22:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:06 smithi094 ceph-mon[31273]: pgmap v305: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:22:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:07 smithi027 ceph-mon[28085]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T21:22:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:07 smithi204 ceph-mon[32145]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T21:22:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:07 smithi094 ceph-mon[31273]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T21:22:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:08 smithi027 ceph-mon[28085]: pgmap v307: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:08 smithi204 ceph-mon[32145]: pgmap v307: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:08.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:08 smithi094 ceph-mon[31273]: pgmap v307: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:10.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:10 smithi027 ceph-mon[28085]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:10 smithi204 ceph-mon[32145]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:10 smithi094 ceph-mon[31273]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:12 smithi027 ceph-mon[28085]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:12 smithi204 ceph-mon[32145]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:12 smithi094 ceph-mon[31273]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:14 smithi204 ceph-mon[32145]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:14 smithi204 ceph-mon[32145]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T21:22:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:14 smithi094 ceph-mon[31273]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:14 smithi094 ceph-mon[31273]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T21:22:14.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:14 smithi027 ceph-mon[28085]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:22:14.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:14 smithi027 ceph-mon[28085]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T21:22:16.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T21:22:16.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 -- 172.21.15.27:0/2806663053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4074d00 msgr2=0x7f02c40750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:16.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 --2- 172.21.15.27:0/2806663053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4074d00 0x7f02c40750e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009900 tx=0x7f02b402a480 comp rx=0 tx=0).stop 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 -- 172.21.15.27:0/2806663053 shutdown_connections 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 --2- 172.21.15.27:0/2806663053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c41113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 --2- 172.21.15.27:0/2806663053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4075620 0x7f02c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 --2- 172.21.15.27:0/2806663053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4074d00 0x7f02c40750e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 -- 172.21.15.27:0/2806663053 >> 172.21.15.27:0/2806663053 conn(0x7f02c40fe270 msgr2=0x7f02c4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 -- 172.21.15.27:0/2806663053 shutdown_connections 2024-04-26T21:22:16.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.725+0000 7f02c8a00640 1 -- 172.21.15.27:0/2806663053 wait complete. 2024-04-26T21:22:16.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 Processor -- start 2024-04-26T21:22:16.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 -- start start 2024-04-26T21:22:16.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4074d00 0x7f02c419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:16.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:16.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4114ac0 con 0x7f02c4075620 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4114940 con 0x7f02c4074d00 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.726+0000 7f02c8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4114c40 con 0x7f02c410efe0 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44450/0 (socket says 172.21.15.27:44450) 2024-04-26T21:22:16.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 -- 172.21.15.27:0/782208086 learned_addr learned my addr 172.21.15.27:0/782208086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02c2800640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4074d00 0x7f02c419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02c3200640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 -- 172.21.15.27:0/782208086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 msgr2=0x7f02c419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 -- 172.21.15.27:0/782208086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4074d00 msgr2=0x7f02c419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4074d00 0x7f02c419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02b9e00640 1 -- 172.21.15.27:0/782208086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c41a0110 con 0x7f02c4075620 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.727+0000 7f02c3200640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02b9e00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac017070 con 0x7f02c4075620 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02ac009c30 con 0x7f02c4075620 2024-04-26T21:22:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c41a0400 con 0x7f02c4075620 2024-04-26T21:22:16.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00f620 con 0x7f02c4075620 2024-04-26T21:22:16.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.728+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41a7c90 con 0x7f02c4075620 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.729+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c4050f10 con 0x7f02c4075620 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.731+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f02ac010460 con 0x7f02c4075620 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.733+0000 7f02bbe00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 0x7f02a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.733+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f02ac0d0580 con 0x7f02c4075620 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.733+0000 7f02c2800640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 0x7f02a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:16.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.734+0000 7f02c2800640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 0x7f02a80a61d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000c00 tx=0x7f02b4005a20 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.738+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f02ac09cb70 con 0x7f02c4075620 2024-04-26T21:22:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:16 smithi204 ceph-mon[32145]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 233 B/s wr, 11 op/s 2024-04-26T21:22:16.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:16 smithi094 ceph-mon[31273]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 233 B/s wr, 11 op/s 2024-04-26T21:22:16.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.971+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f02c4062d70 con 0x7f02a80a3d10 2024-04-26T21:22:16.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:16 smithi027 ceph-mon[28085]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 233 B/s wr, 11 op/s 2024-04-26T21:22:16.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.990+0000 7f02bbe00640 1 -- 172.21.15.27:0/782208086 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f02c4062d70 con 0x7f02a80a3d10 2024-04-26T21:22:16.992 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T21:12:14", "first": "2024-04-26T21:13:00", "last": "2024-04-26T21:22:00", "last_pruned": "2024-04-26T21:22:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-26T21:22:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 msgr2=0x7f02a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 0x7f02a80a61d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000c00 tx=0x7f02b4005a20 comp rx=0 tx=0).stop 2024-04-26T21:22:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 msgr2=0x7f02c419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00de10 comp rx=0 tx=0).stop 2024-04-26T21:22:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 shutdown_connections 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f02a80a3d10 0x7f02a80a61d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02c410efe0 0x7f02c419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02c4075620 0x7f02c419b660 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 --2- 172.21.15.27:0/782208086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4074d00 0x7f02c419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 >> 172.21.15.27:0/782208086 conn(0x7f02c40fe270 msgr2=0x7f02c40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 shutdown_connections 2024-04-26T21:22:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:16.994+0000 7f02c8a00640 1 -- 172.21.15.27:0/782208086 wait complete. 2024-04-26T21:22:17.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 -- 172.21.15.27:0/2133375176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f4102910 msgr2=0x7f40f4102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 --2- 172.21.15.27:0/2133375176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f4102910 0x7f40f4102d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009930 tx=0x7f40e002a420 comp rx=0 tx=0).stop 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 -- 172.21.15.27:0/2133375176 shutdown_connections 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 --2- 172.21.15.27:0/2133375176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f41032d0 0x7f40f410f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 --2- 172.21.15.27:0/2133375176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f4102910 0x7f40f4102d90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 --2- 172.21.15.27:0/2133375176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4108920 0x7f40f4108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 -- 172.21.15.27:0/2133375176 >> 172.21.15.27:0/2133375176 conn(0x7f40f40fe3e0 msgr2=0x7f40f4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 -- 172.21.15.27:0/2133375176 shutdown_connections 2024-04-26T21:22:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.227+0000 7f40fa000640 1 -- 172.21.15.27:0/2133375176 wait complete. 2024-04-26T21:22:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 Processor -- start 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 -- start start 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4102910 0x7f40f419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f4108920 0x7f40f419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112ae0 con 0x7f40f41032d0 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112960 con 0x7f40f4108920 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.228+0000 7f40fa000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112c60 con 0x7f40f4102910 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44470/0 (socket says 172.21.15.27:44470) 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 -- 172.21.15.27:0/4107507392 learned_addr learned my addr 172.21.15.27:0/4107507392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f3400640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4102910 0x7f40f419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f3e00640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f4108920 0x7f40f419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 -- 172.21.15.27:0/4107507392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4102910 msgr2=0x7f40f419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4102910 0x7f40f419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 -- 172.21.15.27:0/4107507392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f4108920 msgr2=0x7f40f419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f4108920 0x7f40f419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.229+0000 7f40f2a00640 1 -- 172.21.15.27:0/4107507392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f41a0230 con 0x7f40f41032d0 2024-04-26T21:22:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40f2a00640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a940 tx=0x7f40e0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e003f070 con 0x7f40f41032d0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40e0003e10 con 0x7f40f41032d0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0035030 con 0x7f40f41032d0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41a04c0 con 0x7f40f41032d0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.230+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41a7db0 con 0x7f40f41032d0 2024-04-26T21:22:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.231+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4104050 con 0x7f40f41032d0 2024-04-26T21:22:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.235+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f40e0042070 con 0x7f40f41032d0 2024-04-26T21:22:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.237+0000 7f40ebe00640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 0x7f40d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.237+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f40e00f1480 con 0x7f40f41032d0 2024-04-26T21:22:17.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.237+0000 7f40f3400640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 0x7f40d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.238+0000 7f40f3400640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 0x7f40d80a6300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005fd0 tx=0x7f40e400b510 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.239+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f40e00bda70 con 0x7f40f41032d0 2024-04-26T21:22:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.472+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 --> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f40f40ffa30 con 0x7f40d80a3e40 2024-04-26T21:22:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.488+0000 7f40ebe00640 1 -- 172.21.15.27:0/4107507392 <== mgr.15342 v2:172.21.15.27:6800/1450148262 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f40f40ffa30 con 0x7f40d80a3e40 2024-04-26T21:22:17.489 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-04-26T21:22:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 msgr2=0x7f40d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 0x7f40d80a6300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005fd0 tx=0x7f40e400b510 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 msgr2=0x7f40f419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a940 tx=0x7f40e0003c20 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 shutdown_connections 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f40d80a3e40 0x7f40d80a6300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.490+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f4108920 0x7f40f419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.491+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f41032d0 0x7f40f419b780 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.491+0000 7f40fa000640 1 --2- 172.21.15.27:0/4107507392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40f4102910 0x7f40f419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.491+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 >> 172.21.15.27:0/4107507392 conn(0x7f40f40fe3e0 msgr2=0x7f40f410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.491+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 shutdown_connections 2024-04-26T21:22:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.491+0000 7f40fa000640 1 -- 172.21.15.27:0/4107507392 wait complete. 2024-04-26T21:22:17.513 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T21:22:17.525 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 -- 172.21.15.27:0/3016524924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806ad20 msgr2=0x7f5e1806b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3016524924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806ad20 0x7f5e1806b160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009cf0 tx=0x7f5e0002ed00 comp rx=0 tx=0).stop 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 -- 172.21.15.27:0/3016524924 shutdown_connections 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3016524924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806b6a0 0x7f5e1810f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3016524924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806ad20 0x7f5e1806b160 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3016524924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 0x7f5e18108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.744+0000 7f5e1f600640 1 -- 172.21.15.27:0/3016524924 >> 172.21.15.27:0/3016524924 conn(0x7f5e18078960 msgr2=0x7f5e18078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.745+0000 7f5e1f600640 1 -- 172.21.15.27:0/3016524924 shutdown_connections 2024-04-26T21:22:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.745+0000 7f5e1f600640 1 -- 172.21.15.27:0/3016524924 wait complete. 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 Processor -- start 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 -- start start 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 0x7f5e18196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 0x7f5e1819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18110880 con 0x7f5e1806b6a0 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18110700 con 0x7f5e18108920 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e1f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18110a00 con 0x7f5e1806ad20 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.746+0000 7f5e17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44490/0 (socket says 172.21.15.27:44490) 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 -- 172.21.15.27:0/3333570863 learned_addr learned my addr 172.21.15.27:0/3333570863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e1cc00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 0x7f5e18196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e1d600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 0x7f5e1819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 -- 172.21.15.27:0/3333570863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 msgr2=0x7f5e18196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 0x7f5e18196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 -- 172.21.15.27:0/3333570863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 msgr2=0x7f5e1819b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 0x7f5e1819b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 -- 172.21.15.27:0/3333570863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1819be40 con 0x7f5e1806b6a0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e1cc00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 0x7f5e18196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.747+0000 7f5e17e00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000f80 tx=0x7f5e00003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.748+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0003c040 con 0x7f5e1806b6a0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.748+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e00047070 con 0x7f5e1806b6a0 2024-04-26T21:22:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.748+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00037850 con 0x7f5e1806b6a0 2024-04-26T21:22:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.748+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1819c0d0 con 0x7f5e1806b6a0 2024-04-26T21:22:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.748+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e181a39c0 con 0x7f5e1806b6a0 2024-04-26T21:22:17.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.749+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e18068f30 con 0x7f5e1806b6a0 2024-04-26T21:22:17.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.751+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5e000379f0 con 0x7f5e1806b6a0 2024-04-26T21:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.752+0000 7f5e15600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 0x7f5dfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.753+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f5e000f5660 con 0x7f5e1806b6a0 2024-04-26T21:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.753+0000 7f5e1cc00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 0x7f5dfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.753+0000 7f5e1cc00640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 0x7f5dfc0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005fd0 tx=0x7f5e080094d0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.757+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e000c1df0 con 0x7f5e1806b6a0 2024-04-26T21:22:17.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.966+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e1810ae60 con 0x7f5e1806b6a0 2024-04-26T21:22:17.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.970+0000 7f5e15600640 1 -- 172.21.15.27:0/3333570863 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23197 (secure 0 0 0) 0x7f5e000c16f0 con 0x7f5e1806b6a0 2024-04-26T21:22:17.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:22:17.971 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:22:13.764482+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/542903082":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6800/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6801/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/2356002952":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/137865064":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/1196187505":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/238118775":"2024-04-27T21:12:12.895075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 msgr2=0x7f5dfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 0x7f5dfc0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005fd0 tx=0x7f5e080094d0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 msgr2=0x7f5e18197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000f80 tx=0x7f5e00003f10 comp rx=0 tx=0).stop 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 shutdown_connections 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5dfc0a3d20 0x7f5dfc0a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e18108920 0x7f5e1819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e1806b6a0 0x7f5e18197390 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 --2- 172.21.15.27:0/3333570863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e1806ad20 0x7f5e18196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.971+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 >> 172.21.15.27:0/3333570863 conn(0x7f5e18078960 msgr2=0x7f5e180ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.972+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 shutdown_connections 2024-04-26T21:22:17.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:17.972+0000 7f5e1f600640 1 -- 172.21.15.27:0/3333570863 wait complete. 2024-04-26T21:22:17.995 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:22:17.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 -- 172.21.15.27:0/4138051618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa958077780 msgr2=0x7fa958113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 --2- 172.21.15.27:0/4138051618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa958077780 0x7fa958113270 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa944009900 tx=0x7fa94402a410 comp rx=0 tx=0).stop 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 -- 172.21.15.27:0/4138051618 shutdown_connections 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 --2- 172.21.15.27:0/4138051618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9581137b0 0x7fa958115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 --2- 172.21.15.27:0/4138051618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa958077780 0x7fa958113270 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 --2- 172.21.15.27:0/4138051618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958076e60 0x7fa958077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.213+0000 7fa95fe00640 1 -- 172.21.15.27:0/4138051618 >> 172.21.15.27:0/4138051618 conn(0x7fa9581005a0 msgr2=0x7fa9581029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.214+0000 7fa95fe00640 1 -- 172.21.15.27:0/4138051618 shutdown_connections 2024-04-26T21:22:18.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.214+0000 7fa95fe00640 1 -- 172.21.15.27:0/4138051618 wait complete. 2024-04-26T21:22:18.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 Processor -- start 2024-04-26T21:22:18.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 -- start start 2024-04-26T21:22:18.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958076e60 0x7fa95819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958077780 0x7fa95819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581190c0 con 0x7fa9581137b0 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa958118f40 con 0x7fa958077780 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.215+0000 7fa95fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa958119240 con 0x7fa958076e60 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44510/0 (socket says 172.21.15.27:44510) 2024-04-26T21:22:18.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 -- 172.21.15.27:0/1743225427 learned_addr learned my addr 172.21.15.27:0/1743225427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95d400640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958076e60 0x7fa95819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95ca00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958077780 0x7fa95819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 -- 172.21.15.27:0/1743225427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958076e60 msgr2=0x7fa95819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958076e60 0x7fa95819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 -- 172.21.15.27:0/1743225427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958077780 msgr2=0x7fa95819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958077780 0x7fa95819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 -- 172.21.15.27:0/1743225427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9581a4670 con 0x7fa9581137b0 2024-04-26T21:22:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.216+0000 7fa95de00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9d0 tx=0x7fa95000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.217+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000cd80 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.217+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa950005020 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.217+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9581a4960 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.217+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500116d0 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.217+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581ac1f0 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.218+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958069750 con 0x7fa9581137b0 2024-04-26T21:22:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.220+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa950004a40 con 0x7fa9581137b0 2024-04-26T21:22:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.222+0000 7fa956000640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 0x7fa9400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.222+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fa9500d0cb0 con 0x7fa9581137b0 2024-04-26T21:22:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.222+0000 7fa95d400640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 0x7fa9400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.223+0000 7fa95d400640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 0x7fa9400a6260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009b60 tx=0x7fa94c005f90 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.226+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa950016030 con 0x7fa9581137b0 2024-04-26T21:22:18.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.433+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa95802dad0 con 0x7fa9581137b0 2024-04-26T21:22:18.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.434+0000 7fa956000640 1 -- 172.21.15.27:0/1743225427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fa95009d300 con 0x7fa9581137b0 2024-04-26T21:22:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.436+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 msgr2=0x7fa9400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.436+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 0x7fa9400a6260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009b60 tx=0x7fa94c005f90 comp rx=0 tx=0).stop 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.436+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 msgr2=0x7fa9581a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.436+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9d0 tx=0x7fa95000ee90 comp rx=0 tx=0).stop 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 shutdown_connections 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa9400a3da0 0x7fa9400a6260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9581137b0 0x7fa9581a3f40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa958077780 0x7fa95819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 --2- 172.21.15.27:0/1743225427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958076e60 0x7fa95819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 >> 172.21.15.27:0/1743225427 conn(0x7fa9581005a0 msgr2=0x7fa958102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 shutdown_connections 2024-04-26T21:22:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.437+0000 7fa95fe00640 1 -- 172.21.15.27:0/1743225427 wait complete. 2024-04-26T21:22:18.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 -- 172.21.15.27:0/3930548870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df41027d0 msgr2=0x7f4df4102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 --2- 172.21.15.27:0/3930548870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df41027d0 0x7f4df4102c50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009930 tx=0x7f4de002a440 comp rx=0 tx=0).stop 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 -- 172.21.15.27:0/3930548870 shutdown_connections 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 --2- 172.21.15.27:0/3930548870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df4103190 0x7f4df410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 --2- 172.21.15.27:0/3930548870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df41027d0 0x7f4df4102c50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 --2- 172.21.15.27:0/3930548870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 0x7f4df4108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.679+0000 7f4dfac00640 1 -- 172.21.15.27:0/3930548870 >> 172.21.15.27:0/3930548870 conn(0x7f4df40fe300 msgr2=0x7f4df4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.680+0000 7f4dfac00640 1 -- 172.21.15.27:0/3930548870 shutdown_connections 2024-04-26T21:22:18.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.680+0000 7f4dfac00640 1 -- 172.21.15.27:0/3930548870 wait complete. 2024-04-26T21:22:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 Processor -- start 2024-04-26T21:22:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 -- start start 2024-04-26T21:22:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df41027d0 0x7f4df419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 0x7f4df419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4112960 con 0x7f4df4103190 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4df41127e0 con 0x7f4df41087e0 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.681+0000 7f4dfac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4112ae0 con 0x7f4df41027d0 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44526/0 (socket says 172.21.15.27:44526) 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 -- 172.21.15.27:0/17212578 learned_addr learned my addr 172.21.15.27:0/17212578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4dfa200640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 0x7f4df419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df9800640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df41027d0 0x7f4df419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 -- 172.21.15.27:0/17212578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df41027d0 msgr2=0x7f4df419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df41027d0 0x7f4df419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 -- 172.21.15.27:0/17212578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 msgr2=0x7f4df419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 0x7f4df419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 -- 172.21.15.27:0/17212578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df41a00c0 con 0x7f4df4103190 2024-04-26T21:22:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.682+0000 7f4df8e00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4de002a960 tx=0x7f4de0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.683+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de003f070 con 0x7f4df4103190 2024-04-26T21:22:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.683+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4de0003e10 con 0x7f4df4103190 2024-04-26T21:22:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.683+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0035030 con 0x7f4df4103190 2024-04-26T21:22:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.683+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a0350 con 0x7f4df4103190 2024-04-26T21:22:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.683+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41a7c40 con 0x7f4df4103190 2024-04-26T21:22:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.684+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df4069670 con 0x7f4df4103190 2024-04-26T21:22:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.686+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4de0042070 con 0x7f4df4103190 2024-04-26T21:22:18.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.687+0000 7f4df2000640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 0x7f4ddc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:18.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.688+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f4de00f10a0 con 0x7f4df4103190 2024-04-26T21:22:18.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.688+0000 7f4df9800640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 0x7f4ddc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:18.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.689+0000 7f4df9800640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 0x7f4ddc0a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005fd0 tx=0x7f4de8005850 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:18.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.691+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4de00bd690 con 0x7f4df4103190 2024-04-26T21:22:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:18 smithi027 ceph-mon[28085]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:18 smithi027 ceph-mon[28085]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:22:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:18 smithi027 ceph-mon[28085]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:18 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3333570863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:22:18.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:18 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1743225427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:22:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:18 smithi204 ceph-mon[32145]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:18 smithi204 ceph-mon[32145]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:22:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:18 smithi204 ceph-mon[32145]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:18 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3333570863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:22:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:18 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1743225427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:22:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:18 smithi094 ceph-mon[31273]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:18 smithi094 ceph-mon[31273]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-26T21:22:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:18 smithi094 ceph-mon[31273]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:18 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3333570863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:22:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:18 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1743225427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:22:18.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.898+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4df41a7e10 con 0x7f4df4103190 2024-04-26T21:22:18.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.899+0000 7f4df2000640 1 -- 172.21.15.27:0/17212578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f4de00bcf90 con 0x7f4df4103190 2024-04-26T21:22:18.901 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.901+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 msgr2=0x7f4ddc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.901+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 0x7f4ddc0a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005fd0 tx=0x7f4de8005850 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.901+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 msgr2=0x7f4df419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4de002a960 tx=0x7f4de0003c20 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 shutdown_connections 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4ddc0a3da0 0x7f4ddc0a6260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4df41087e0 0x7f4df419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df4103190 0x7f4df419b610 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 --2- 172.21.15.27:0/17212578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4df41027d0 0x7f4df419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 >> 172.21.15.27:0/17212578 conn(0x7f4df40fe300 msgr2=0x7f4df410dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 shutdown_connections 2024-04-26T21:22:18.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:18.902+0000 7f4dfac00640 1 -- 172.21.15.27:0/17212578 wait complete. 2024-04-26T21:22:18.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:22:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 -- 172.21.15.27:0/2446505174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc738104900 msgr2=0x7fc738104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 --2- 172.21.15.27:0/2446505174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc738104900 0x7fc738104d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc724009930 tx=0x7fc72402a440 comp rx=0 tx=0).stop 2024-04-26T21:22:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 -- 172.21.15.27:0/2446505174 shutdown_connections 2024-04-26T21:22:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 --2- 172.21.15.27:0/2446505174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7381052c0 0x7fc7381117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 --2- 172.21.15.27:0/2446505174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc738104900 0x7fc738104d80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 --2- 172.21.15.27:0/2446505174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc73810a910 0x7fc73810acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 -- 172.21.15.27:0/2446505174 >> 172.21.15.27:0/2446505174 conn(0x7fc7381003d0 msgr2=0x7fc7381027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 -- 172.21.15.27:0/2446505174 shutdown_connections 2024-04-26T21:22:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.154+0000 7fc73e400640 1 -- 172.21.15.27:0/2446505174 wait complete. 2024-04-26T21:22:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.155+0000 7fc73e400640 1 Processor -- start 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.155+0000 7fc73e400640 1 -- start start 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.155+0000 7fc73e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc738104900 0x7fc73819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.155+0000 7fc73e400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 0x7fc73819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73e400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc738114ad0 con 0x7fc7381052c0 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc738114950 con 0x7fc73810a910 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc738114c50 con 0x7fc738104900 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52488/0 (socket says 172.21.15.27:52488) 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 -- 172.21.15.27:0/3690351640 learned_addr learned my addr 172.21.15.27:0/3690351640 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73d000640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc738104900 0x7fc73819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73da00640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 0x7fc73819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 -- 172.21.15.27:0/3690351640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc738104900 msgr2=0x7fc73819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc738104900 0x7fc73819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 -- 172.21.15.27:0/3690351640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 msgr2=0x7fc73819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 0x7fc73819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc737e00640 1 -- 172.21.15.27:0/3690351640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73819ff40 con 0x7fc7381052c0 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.156+0000 7fc73da00640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 0x7fc73819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc737e00640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc72402a960 tx=0x7fc724003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72403f070 con 0x7fc7381052c0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc72403bdc0 con 0x7fc7381052c0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7381a01d0 con 0x7fc7381052c0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7381a7ac0 con 0x7fc7381052c0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.157+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724049060 con 0x7fc7381052c0 2024-04-26T21:22:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.158+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc738069ac0 con 0x7fc7381052c0 2024-04-26T21:22:19.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.160+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc724042070 con 0x7fc7381052c0 2024-04-26T21:22:19.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.162+0000 7fc735600640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.162+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fc7240f1280 con 0x7fc7381052c0 2024-04-26T21:22:19.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.162+0000 7fc73d000640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.163+0000 7fc73d000640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 0x7fc7200a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00cac0 tx=0x7fc72c00c4a0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:19.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.167+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc7240bd870 con 0x7fc7381052c0 2024-04-26T21:22:19.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.375+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc73810c8e0 con 0x7fc7381052c0 2024-04-26T21:22:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.376+0000 7fc735600640 1 -- 172.21.15.27:0/3690351640 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fc7240bd170 con 0x7fc7381052c0 2024-04-26T21:22:19.377 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:22:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 msgr2=0x7fc7200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 0x7fc7200a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00cac0 tx=0x7fc72c00c4a0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 msgr2=0x7fc73819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc72402a960 tx=0x7fc724003d60 comp rx=0 tx=0).stop 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 shutdown_connections 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc73810a910 0x7fc73819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7381052c0 0x7fc73819b490 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 --2- 172.21.15.27:0/3690351640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc738104900 0x7fc73819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 >> 172.21.15.27:0/3690351640 conn(0x7fc7381003d0 msgr2=0x7fc738100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 shutdown_connections 2024-04-26T21:22:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.379+0000 7fc73e400640 1 -- 172.21.15.27:0/3690351640 wait complete. 2024-04-26T21:22:19.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.625+0000 7fac85c00640 1 -- 172.21.15.27:0/60992598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 msgr2=0x7fac800750c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.625+0000 7fac85c00640 1 --2- 172.21.15.27:0/60992598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac800750c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac70009900 tx=0x7fac7002a480 comp rx=0 tx=0).stop 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.625+0000 7fac85c00640 1 -- 172.21.15.27:0/60992598 shutdown_connections 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 --2- 172.21.15.27:0/60992598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac8010efd0 0x7fac801113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 --2- 172.21.15.27:0/60992598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80075600 0x7fac8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 --2- 172.21.15.27:0/60992598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac800750c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 -- 172.21.15.27:0/60992598 >> 172.21.15.27:0/60992598 conn(0x7fac800fe260 msgr2=0x7fac80100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 -- 172.21.15.27:0/60992598 shutdown_connections 2024-04-26T21:22:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.626+0000 7fac85c00640 1 -- 172.21.15.27:0/60992598 wait complete. 2024-04-26T21:22:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 Processor -- start 2024-04-26T21:22:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 -- start start 2024-04-26T21:22:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80075600 0x7fac8019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 0x7fac8019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac80114a50 con 0x7fac80074ce0 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac801148d0 con 0x7fac8010efd0 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.627+0000 7fac85c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac80114bd0 con 0x7fac80075600 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52514/0 (socket says 172.21.15.27:52514) 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 -- 172.21.15.27:0/2474097659 learned_addr learned my addr 172.21.15.27:0/2474097659 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7f400640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80075600 0x7fac8019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac85200640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 0x7fac8019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 -- 172.21.15.27:0/2474097659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80075600 msgr2=0x7fac8019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80075600 0x7fac8019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 -- 172.21.15.27:0/2474097659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 msgr2=0x7fac8019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 0x7fac8019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac7fe00640 1 -- 172.21.15.27:0/2474097659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801a0060 con 0x7fac80074ce0 2024-04-26T21:22:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.628+0000 7fac85200640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 0x7fac8019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac7fe00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac70009230 tx=0x7fac70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7003f030 con 0x7fac80074ce0 2024-04-26T21:22:19.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac70003a80 con 0x7fac80074ce0 2024-04-26T21:22:19.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70035070 con 0x7fac80074ce0 2024-04-26T21:22:19.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801a02f0 con 0x7fac80074ce0 2024-04-26T21:22:19.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.629+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a7be0 con 0x7fac80074ce0 2024-04-26T21:22:19.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.630+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac80069ac0 con 0x7fac80074ce0 2024-04-26T21:22:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.632+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fac70042030 con 0x7fac80074ce0 2024-04-26T21:22:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.634+0000 7fac7cc00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 0x7fac640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.634+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fac700f1440 con 0x7fac80074ce0 2024-04-26T21:22:19.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.634+0000 7fac7f400640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 0x7fac640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:19.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.635+0000 7fac7f400640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 0x7fac640a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac80100a30 tx=0x7fac6c005f90 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:19.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.636+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac700bda30 con 0x7fac80074ce0 2024-04-26T21:22:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/17212578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:22:19.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3690351640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:22:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/17212578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:22:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3690351640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:22:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.898+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fac8010a150 con 0x7fac80074ce0 2024-04-26T21:22:19.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.929+0000 7fac7cc00640 1 -- 172.21.15.27:0/2474097659 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746061 (secure 0 0 0) 0x7fac700bd330 con 0x7fac80074ce0 2024-04-26T21:22:19.942 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 25, 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 15342, 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1450148262 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1450148262 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/1450148262", 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-04-26T21:12:12.895385+0000", 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 25006, 2024-04-26T21:22:19.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.948 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:19.949 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T21:22:19.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:19.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:19.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T21:22:19.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:22:19.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:19.967 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T21:22:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:19.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:19.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:19.978 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T21:22:19.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:19.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T21:22:19.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T21:22:19.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:19.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.985 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.986 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T21:22:19.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T21:22:19.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T21:22:19.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:22:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:19.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.002 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T21:22:20.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T21:22:20.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:22:20.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T21:22:20.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T21:22:20.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.018 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.020 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.021 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.028 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T21:22:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:22:20.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.046 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T21:22:20.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:22:20.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:22:20.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:22:20.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:22:20.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:22:20.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:22:20.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:22:20.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:22:20.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:22:20.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.089 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:22:20.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:22:20.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.099 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:22:20.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:20.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T21:22:20.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.114 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T21:22:20.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.119 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T21:22:20.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T21:22:20.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T21:22:20.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.125 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.126 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T21:22:20.133 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.136 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.140 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T21:22:20.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.149 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.151 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.154 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.156 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:22:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.161 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:22:20.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.167 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.170 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T21:22:20.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T21:22:20.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.176 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.181 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.182 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T21:22:20.198 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.199 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.200 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T21:22:20.204 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.205 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.217 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T21:22:20.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:22:20.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T21:22:20.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:22:20.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.231 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T21:22:20.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T21:22:20.243 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.244 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T21:22:20.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.254 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.259 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.261 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.267 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.273 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.281 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.282 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T21:22:20.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T21:22:20.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T21:22:20.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.293 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.295 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.296 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.298 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.300 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.301 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.308 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:22:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T21:22:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T21:22:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T21:22:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 35030, 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T21:22:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T21:22:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T21:22:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:22:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T21:22:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T21:22:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T21:22:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:22:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T21:22:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T21:22:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T21:22:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:22:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:22:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:22:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:22:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T21:22:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:22:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:22:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:22:20.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T21:22:20.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:22:20.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T21:22:20.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T21:22:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:22:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:22:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:22:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:22:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:22:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:22:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:22:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:22:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T21:22:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T21:22:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T21:22:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T21:22:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T21:22:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T21:22:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.569 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.573 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.576 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T21:22:20.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.579 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T21:22:20.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.592 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.595 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.596 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:22:20.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.601 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.604 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T21:22:20.606 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:22:20.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.608 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.611 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T21:22:20.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.614 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:22:20.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.622 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.630 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.634 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.637 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.639 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.644 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:20.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T21:22:20.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.649 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.655 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.658 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.660 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.661 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.664 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T21:22:20.668 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:22:20.669 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.672 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.674 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:22:20.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.679 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:22:20.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.685 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:20.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.694 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:20.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.697 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.698 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.701 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:22:20.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:22:20.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:20.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.709 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.710 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.713 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:20.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.716 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.719 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.728 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.733 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.735 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T21:22:20.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.744 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T21:22:20.751 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.754 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.755 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.764 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.766 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.767 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.769 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.771 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.774 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.776 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.778 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.779 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T21:22:20.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.787 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.798 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.805 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.809 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.812 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.815 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:22:20.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T21:22:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.826 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.829 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.831 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T21:22:20.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T21:22:20.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T21:22:20.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T21:22:20.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-04-26T21:22:20.837 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:20.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.840 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.843 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-04-26T21:22:20.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.846 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-04-26T21:22:20.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-04-26T21:22:20.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.856 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:22:20.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-04-26T21:22:20.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.862 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.865 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.868 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-04-26T21:22:20.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-04-26T21:22:20.872 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:20.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.875 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.878 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:22:20.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.887 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-04-26T21:22:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-04-26T21:22:20.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:22:20.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:22:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-04-26T21:22:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:22:20.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-04-26T21:22:20.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-04-26T21:22:20.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:22:20.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:22:20.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-04-26T21:22:20.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.940 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:22:20.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-04-26T21:22:20.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-04-26T21:22:20.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:22:20.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:20.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.953 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:20.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:22:20.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:22:20.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.974 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:22:20.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:22:20.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:22:20.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:22:20.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:20.998 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:20.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:22:21.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:21.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-04-26T21:22:21.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-04-26T21:22:21.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:22:21.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-04-26T21:22:21.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:21.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.037 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.040 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.041 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-04-26T21:22:21.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.048 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:21.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:21.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-04-26T21:22:21.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.056 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.059 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-04-26T21:22:21.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:22:21.065 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:22:21.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.071 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.074 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-04-26T21:22:21.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-04-26T21:22:21.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-04-26T21:22:21.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-04-26T21:22:21.083 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.084 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.087 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-04-26T21:22:21.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.098 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.099 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.102 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:22:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-04-26T21:22:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-04-26T21:22:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-04-26T21:22:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-04-26T21:22:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-04-26T21:22:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-04-26T21:22:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-04-26T21:22:21.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.193 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.203 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:22:21.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.206 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-04-26T21:22:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.212 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.215 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.220 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.221 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.223 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-04-26T21:22:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.226 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.229 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.236 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-04-26T21:22:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.249 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.264 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.269 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.270 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-04-26T21:22:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-04-26T21:22:21.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-04-26T21:22:21.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:22:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-04-26T21:22:21.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.294 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-04-26T21:22:21.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.305 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.309 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.315 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.318 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:22:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:22:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-04-26T21:22:21.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-04-26T21:22:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-04-26T21:22:21.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.329 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-04-26T21:22:21.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-04-26T21:22:21.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.337 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T21:22:21.338 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T21:22:21.339 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T21:22:21.340 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T21:22:21.341 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-04-26T21:22:21.342 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 91, 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3569133302 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2633945635 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3838922220 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 353646035 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 717691800 2024-04-26T21:22:21.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3743093853 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:21.347 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-26T21:22:21.348 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:22:21.349 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:20 smithi094 ceph-mon[31273]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:21.349 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:20 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2474097659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:22:21.350 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/17212578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:22:21.350 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3690351640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:22:21.350 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:20 smithi204 ceph-mon[32145]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:21.350 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:20 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2474097659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:22:21.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 msgr2=0x7fac640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:21.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 0x7fac640a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac80100a30 tx=0x7fac6c005f90 comp rx=0 tx=0).stop 2024-04-26T21:22:21.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 msgr2=0x7fac8019b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac70009230 tx=0x7fac70004270 comp rx=0 tx=0).stop 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 shutdown_connections 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fac640a3d20 0x7fac640a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac80075600 0x7fac8019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac8010efd0 0x7fac8019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 --2- 172.21.15.27:0/2474097659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac80074ce0 0x7fac8019b070 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 >> 172.21.15.27:0/2474097659 conn(0x7fac800fe260 msgr2=0x7fac800ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 shutdown_connections 2024-04-26T21:22:21.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:22:19.940+0000 7fac85c00640 1 -- 172.21.15.27:0/2474097659 wait complete. 2024-04-26T21:22:21.357 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T21:22:21.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T21:22:21.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:20 smithi027 ceph-mon[28085]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:21.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:20 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2474097659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:22:21.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:21 smithi027 ceph-mon[28085]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:21.866 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:22.002 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:21 smithi204 ceph-mon[32145]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:22.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:21 smithi094 ceph-mon[31273]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:24.271 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:22:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 416, 2024-04-26T21:22:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 693, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 146416, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 557389, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 11, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.164393048, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.080714867, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.065593645, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.007392773, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 692, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.126757411, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000183175 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 902, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000477049, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000528 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 309088 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 78, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 128, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 26800, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 221913, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 16, 2024-04-26T21:22:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.040835259, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.016497569, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.021838365, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000247002, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 125, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.059829293, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000478634 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 117, 2024-04-26T21:22:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000180033, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000001538 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 140336 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 335, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 323, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1522008, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 164613, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 8, 2024-04-26T21:22:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.118973740, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.044939536, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.084226216, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.010697484, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 322, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.065991933, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000204943 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 84, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000129583, 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000001542 2024-04-26T21:22:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1497408, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 136448 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 55, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.002952158, 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000053675 2024-04-26T21:22:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000006775, 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000003387 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-04-26T21:22:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 4, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 3, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001299786, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000433262 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 1, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 11, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-04-26T21:22:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 16, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 16, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 13, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 3, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 3, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 14, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 7, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 1, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 4, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-04-26T21:22:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 4, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 234, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-04-26T21:22:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 9, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 6, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 8, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 7, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 7, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 2, 2024-04-26T21:22:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4196449, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 6, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.032176075, 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.005362679 2024-04-26T21:22:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 20, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 21, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 1, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 19, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 20, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 1, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 16, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 19, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 3, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 2, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 2, 2024-04-26T21:22:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 51072, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 223608 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 5, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 4, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 17, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-04-26T21:22:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000260707, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000260707 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001039079, 2024-04-26T21:22:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000519539 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 2, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 1, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 2, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 611, 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 279479, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 100, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 191, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 90213, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 346, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-04-26T21:22:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 44, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 12592, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 33, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 44, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 44, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.483987534, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.010999716 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 44, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 68 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 44, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 44, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 6, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 7, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-04-26T21:22:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 6, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-04-26T21:22:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-04-26T21:22:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 99, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 11, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 9, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 9, 2024-04-26T21:22:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 17, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 0, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 2 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 825, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1604019, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 825, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 825, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1604019, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 44, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 12592, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 44, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 12592, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 44, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 44, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 44, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 44, 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x55b81040c500": { 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 7, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 2145, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 7, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 6, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2145, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:24.318 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:22:24.319 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:24 smithi027 ceph-mon[28085]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:24.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:24 smithi094 ceph-mon[31273]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:24.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:24 smithi204 ceph-mon[32145]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-26T21:22:25.252 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=9 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:25.253 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:25.253 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T21:22:25.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T21:22:25.733 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:26.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:26 smithi027 ceph-mon[28085]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 89 B/s wr, 15 op/s 2024-04-26T21:22:26.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:26 smithi094 ceph-mon[31273]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 89 B/s wr, 15 op/s 2024-04-26T21:22:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:26 smithi204 ceph-mon[32145]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 89 B/s wr, 15 op/s 2024-04-26T21:22:28.173 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:22:28.173 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.004260143, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:22:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000729557, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.175 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 229, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 217, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1454992, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 132832, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.103602804, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.035981898, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.076887564, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.011056431, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 217, 2024-04-26T21:22:28.176 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.022685823, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000104542 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1454992, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 132832 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 164, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-04-26T21:22:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 185872, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.034936048, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.025381479, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.004278291, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 161, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.027292205, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000169516 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:28.178 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 185872 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:28.180 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:28.181 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148732, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 31, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:28.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 227, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1420341, 2024-04-26T21:22:28.183 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 227, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 227, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1420341, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:28.184 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:22:28.629 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:28 smithi027 ceph-mon[28085]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T21:22:28.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:28 smithi204 ceph-mon[32145]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T21:22:28.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:28 smithi094 ceph-mon[31273]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T21:22:29.058 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T21:22:29.058 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T21:22:29.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T21:22:29.548 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:30.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:30 smithi027 ceph-mon[28085]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T21:22:30.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:30 smithi204 ceph-mon[32145]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T21:22:30.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:30 smithi094 ceph-mon[31273]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T21:22:31.805 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:22:31.805 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:31.805 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 766, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 765, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 259003, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 242447, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.213857601, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.093388124, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.048320211, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.048430770, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 764, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.085449256, 2024-04-26T21:22:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000111844 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 1386, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 1545, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 355153, 2024-04-26T21:22:31.807 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 888089, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 7, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.277185054, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.105197897, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.068164360, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.077269989, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1542, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.154305449, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000100068 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.808 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 447920 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 240, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 229, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1481284, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 132917, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.099123879, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.036109550, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.069916752, 2024-04-26T21:22:31.809 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.010518715, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 228, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.025692704, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000112687 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1479936, 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 131952 2024-04-26T21:22:31.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3654, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.199862992, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000054697 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-04-26T21:22:31.811 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-04-26T21:22:31.812 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-04-26T21:22:31.813 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 1, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:31.814 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:31.815 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-04-26T21:22:31.816 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-04-26T21:22:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-04-26T21:22:31.818 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 7, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4196449, 2024-04-26T21:22:31.819 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 4196308, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 7 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 3, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 3, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 0, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 1, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 1, 2024-04-26T21:22:31.820 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 49644, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 215416 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-04-26T21:22:31.821 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.822 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.823 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-04-26T21:22:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.827 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.828 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.829 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.830 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.831 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 2, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 3, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 4867, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:31.833 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:31.834 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:31.835 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 197473, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 55, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:31.836 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 191, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 8512, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 13, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-04-26T21:22:31.837 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2148, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2148, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2148, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.873926283, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001337954 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.838 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2148, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2149 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2148, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2148, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1379, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 767, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-04-26T21:22:31.839 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:31.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-04-26T21:22:31.841 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 99, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 11, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 6, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 6, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-04-26T21:22:31.842 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-04-26T21:22:31.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2388, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1865460, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2388, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2388, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1865460, 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:31.844 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2148, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2150, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2148, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2150, 2024-04-26T21:22:31.845 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:31.846 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2148, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2148, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2148, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2148, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:31.847 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x55cdec516780": { 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:31.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:31.849 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:22:32.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:32 smithi027 ceph-mon[28085]: pgmap v320: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:32.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:32 smithi204 ceph-mon[32145]: pgmap v320: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:32.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:32 smithi094 ceph-mon[31273]: pgmap v320: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:32.893 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-26T21:22:32.893 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-26T21:22:32.894 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-26T21:22:32.894 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-26T21:22:32.894 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-26T21:22:32.894 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:32.894 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:32.894 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T21:22:32.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T21:22:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:34.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:34 smithi027 ceph-mon[28085]: pgmap v321: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:34.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:34 smithi204 ceph-mon[32145]: pgmap v321: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:34.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:34 smithi094 ceph-mon[31273]: pgmap v321: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 228, 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 218, 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1452864, 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 134304, 2024-04-26T21:22:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.106258007, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.035972529, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.074111252, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.012286464, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 217, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.023765135, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000109516 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1452864, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 134304 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 167, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 186400, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.039315471, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.029391811, 2024-04-26T21:22:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.004116555, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 164, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.030575492, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000186435 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 186400 2024-04-26T21:22:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2944, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.001645284, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000088237, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001482109, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001912501, 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000637500 2024-04-26T21:22:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148794, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 32, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-04-26T21:22:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 230, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1420731, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 230, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 230, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1420731, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-26T21:22:35.831 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-26T21:22:36.688 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:36 smithi027 ceph-mon[28085]: pgmap v322: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:36.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:36 smithi204 ceph-mon[32145]: pgmap v322: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:36.764 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T21:22:36.765 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T21:22:36.765 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T21:22:36.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T21:22:36.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:36 smithi094 ceph-mon[31273]: pgmap v322: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:22:38.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:38 smithi094 ceph-mon[31273]: pgmap v323: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:38.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:38 smithi027 ceph-mon[28085]: pgmap v323: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:38.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:38 smithi204 ceph-mon[32145]: pgmap v323: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2355, 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2146, 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 854960, 2024-04-26T21:22:39.576 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 2046638, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 11, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.524212365, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.242152123, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.161123720, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.101971008, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2151, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.510046181, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000237120 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1736, 2024-04-26T21:22:39.577 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000721999, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000415 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 316048 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3730, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2525, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1292840, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1045313, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 17, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 9, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.712482267, 2024-04-26T21:22:39.578 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.251867431, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.246261571, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.197567099, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2523, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.806571915, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000319687 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2612, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000972030, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000372 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 306640, 2024-04-26T21:22:39.579 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 192656 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1915, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1447, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2100275, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 551321, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 12, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 10, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.419974668, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.146444148, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.199209234, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.089328903, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1439, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.355298331, 2024-04-26T21:22:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000246906 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1290, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000668482, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000518 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1686800, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 173344 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.581 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 394, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.233294069, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000592116 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2902, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.122927995, 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000042359 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 155, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 155, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 155, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.503217461, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.003246564 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 12, 2024-04-26T21:22:39.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 38, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 1, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 37, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 36, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 3, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 3, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 193, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 182, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-26T21:22:39.584 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-26T21:22:39.585 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 6, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 18, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 71, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:39.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 81, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 61, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 5, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 31 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 13, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 31, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 2, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:39.587 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 5, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 3, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 2, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 2, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:39.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 195, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 195, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-26T21:22:39.589 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 5222039, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 157, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.986514527, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.006283532 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T21:22:39.590 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 30, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 33, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 3, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 20, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 20, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 38, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 41, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 3, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 3, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 21, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 18, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 54136, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 231800 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.591 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 206, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 1, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 155, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 31, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 2, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 7, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018931905, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002704557 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 97, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.050566747, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000521306 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.592 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000689761, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000344880 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:39.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.594 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 10, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.004458098, 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000445809 2024-04-26T21:22:39.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 8, 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.033373015, 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.004171626 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.596 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 10, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.217161947, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.021716194 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:39.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.016156916, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.008078458 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.005952190, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002976095 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:39.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.026893171, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.013446585 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.055643500, 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.013910875 2024-04-26T21:22:39.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002321565, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002321565 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 7, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.048846014, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.006978002 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.022222632, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.007407544 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 5, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 3, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 2, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 28, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 14, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 600, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.603 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:39.604 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:39.605 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 364045, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 147, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 616, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 7, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:39.606 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 191, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 158444, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 1013, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T21:22:39.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 3831, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 1045440, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 37, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 3831, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3831, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 55.230103220, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.014416628 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T21:22:39.608 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3831, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3884 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 3831, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 5, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 3826, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 2, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 11, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 3, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 160, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 34, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T21:22:39.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 3584, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 22, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T21:22:39.610 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T21:22:39.611 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 99, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 12, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 12, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 12, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 54, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 14 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T21:22:39.612 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 2560, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 1, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 3072, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 3, 2024-04-26T21:22:39.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 7996, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 3462213, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 7996, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 7996, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 3462213, 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:39.614 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3831, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 1045783, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3831, 2024-04-26T21:22:39.615 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1045783, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:39.616 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3831, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 3831, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3831, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 3831, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 343, 2024-04-26T21:22:39.617 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 343, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55e264db0500": { 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:39.618 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 195, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1027735, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 195, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 157, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1027735, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:39.619 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T21:22:40.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:40 smithi094 ceph-mon[31273]: pgmap v324: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:40.634 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=13 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-26T21:22:40.635 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-26T21:22:40.635 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T21:22:40.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T21:22:40.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:40 smithi027 ceph-mon[28085]: pgmap v324: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:40.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:40 smithi204 ceph-mon[32145]: pgmap v324: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:41.127 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:22:42.645 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:42 smithi094 ceph-mon[31273]: pgmap v325: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:42.645 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:42 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:42.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:42 smithi027 ceph-mon[28085]: pgmap v325: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:42.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:42 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:42.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:42 smithi204 ceph-mon[32145]: pgmap v325: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:42.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:42 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 412, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 391, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1564590, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 195189, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 11, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.167688341, 2024-04-26T21:22:43.184 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.055947178, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.137110323, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.011897236, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 393, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.107949708, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000274681 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 196, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000146016, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000744 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.185 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1519376, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 145520 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 31, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 225, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 6687, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 454895, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 15, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.043265701, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.021212842, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.015370912, 2024-04-26T21:22:43.186 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000467876, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 220, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.056804061, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000258200 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 50, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000029978, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000599 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 386816 2024-04-26T21:22:43.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 429, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 589, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 153115, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 266244, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 8, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.186802489, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.102885851, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.076553459, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008091132, 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:43.188 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 587, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.117757095, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000200608 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 968, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000522961, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000540 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T21:22:43.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 58, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.003073781, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000052996 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000012643, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000006321 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 4, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 2, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001041686, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000520843 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T21:22:43.191 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 2, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 11, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 16, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 16, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 13, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 2, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 1, 2024-04-26T21:22:43.192 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 1, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 3, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 12, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 6, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 4, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T21:22:43.193 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:43.194 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 1, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 2 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 9, 2024-04-26T21:22:43.195 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 6, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 8, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-26T21:22:43.196 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 7, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T21:22:43.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 7, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4196449, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 6, 2024-04-26T21:22:43.198 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.058396864, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.009732810 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 20, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 21, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 1, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 19, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 21, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 2, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 16, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 18, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 2, 2024-04-26T21:22:43.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 1, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 2, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 1, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 51320, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 223608 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 5, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 4, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 17, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.200 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001041686, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000520843 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.201 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T21:22:43.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.204 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.208 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.209 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T21:22:43.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.211 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 3, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 2, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 1, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 627, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:43.213 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:43.214 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 264504, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 98, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:43.215 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 191, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 90549, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 344, 2024-04-26T21:22:43.216 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 43, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 12541, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 33, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 43, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 43, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.600990215, 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.013976516 2024-04-26T21:22:43.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 43, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 66 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 43, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 43, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 10, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 6, 2024-04-26T21:22:43.218 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 7, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 6, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 20, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T21:22:43.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T21:22:43.220 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 99, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 8, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T21:22:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 16, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 1 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 868, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1628588, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 868, 2024-04-26T21:22:43.223 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 868, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1628588, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:43.224 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 43, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 12541, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 43, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 12541, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T21:22:43.225 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 43, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 43, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 43, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 43, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.227 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x556d43256500": { 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 7, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 2145, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 7, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 6, 2024-04-26T21:22:43.228 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2145, 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:43.229 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T21:22:43.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:43 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:43 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:43 smithi204 ceph-mon[32145]: pgmap v326: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:43.973 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:43 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.973 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:43 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.973 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:43 smithi094 ceph-mon[31273]: pgmap v326: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:43.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:43 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:43 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:43.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:43 smithi027 ceph-mon[28085]: pgmap v326: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=9 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:44.039 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:44.040 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T21:22:44.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T21:22:44.879 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:22:46.174 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:45 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.174 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:45 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.174 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:45 smithi094 ceph-mon[31273]: pgmap v327: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:46.174 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:45 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.174 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:45 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:45 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:45 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:45 smithi027 ceph-mon[28085]: pgmap v327: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:46.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:45 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:45 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:45 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:45 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:45 smithi204 ceph-mon[32145]: pgmap v327: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:22:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:45 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:46.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:45 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:47.010 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T21:22:47.010 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 441, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 613, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 157555, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 249816, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 10, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.197035839, 2024-04-26T21:22:47.011 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.106854104, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.077597868, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008440998, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 612, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.138552848, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000226393 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.012 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 970, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000507414, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000523 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 356, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 365, 2024-04-26T21:22:47.013 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1542685, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 222808, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 13, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.136103098, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.045540395, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.100020933, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.011530381, 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:47.014 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 364, 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.083978813, 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000230711 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 143, 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000512231, 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000003582 2024-04-26T21:22:47.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1516832, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 141232 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 64, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 228, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 24278, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 451925, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 11, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 7, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.048905091, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.026379051, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.017551843, 2024-04-26T21:22:47.016 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000414671, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 225, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.063554713, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000282465 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 84, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000341986, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000004071 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 421840 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.017 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 54, 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002482726, 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000045976 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000008838, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000004419 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 2, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 2, 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T21:22:47.019 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.004703512, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002351756 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 11, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 16, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 16, 2024-04-26T21:22:47.020 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 13, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 2, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 2, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 2, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 3, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 10, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 7, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 3, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T21:22:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-26T21:22:47.022 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:47.023 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-26T21:22:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 11, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 7, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 8, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-26T21:22:47.025 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T21:22:47.026 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 5, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 5, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T21:22:47.027 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195917, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.019481968, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.004870492 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 20, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 21, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 1, 2024-04-26T21:22:47.028 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 19, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 22, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 3, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 16, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 18, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 2, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 2, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 2, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 51200, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 223608 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.029 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 5, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 2, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 13, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 1, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:47.030 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.004703512, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002351756 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.031 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.032 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.034 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.035 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T21:22:47.035 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.036 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.036 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T21:22:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.039 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T21:22:47.040 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.040 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.040 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.040 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:47.041 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.045 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T21:22:47.045 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.045 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.046 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.047 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.049 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T21:22:47.050 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.051 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.052 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.053 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-04-26T21:22:47.054 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 1, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 1, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 634, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T21:22:47.055 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:47.056 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:47.057 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:47.058 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:47.059 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 254915, 2024-04-26T21:22:47.060 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 89, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 191, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 90933, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 348, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 41, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 12070, 2024-04-26T21:22:47.062 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 33, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 41, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 41, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.532991324, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.012999788 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 41, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 64 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 41, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 41, 2024-04-26T21:22:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 10, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 4, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 7, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 6, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:47.064 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 20, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T21:22:47.065 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T21:22:47.066 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 99, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 9, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 9, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T21:22:47.067 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 16, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 1 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T21:22:47.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 857, 2024-04-26T21:22:47.069 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1630060, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 857, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 857, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1630060, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.070 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:47.071 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 41, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 12070, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 41, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 12070, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:47.073 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 41, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 41, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 41, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 41, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.074 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.075 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x558d9e378500": { 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 5, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1613, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 5, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 4, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1613, 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:47.076 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:47.077 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:47.077 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:47.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:47.077 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T21:22:47.077 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.078 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:46 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:22:47.236 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.237 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.237 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.237 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:46 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi027 to 185039769: error parsing value: Value '185039769' is below minimum 939524096 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:46 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:47.942 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=11 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:47.943 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:47.943 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T21:22:47.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T21:22:48.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:47 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:22:48.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:47 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:22:48.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:47 smithi027 ceph-mon[28085]: pgmap v328: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:48.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:47 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:47 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:47 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:22:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:47 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:22:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:47 smithi204 ceph-mon[32145]: pgmap v328: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:47 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:47 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:47 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi204 to 176.4M 2024-04-26T21:22:48.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:47 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi204 to 185036185: error parsing value: Value '185036185' is below minimum 939524096 2024-04-26T21:22:48.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:47 smithi094 ceph-mon[31273]: pgmap v328: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:48.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:47 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:47 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:48.373 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:49.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:49 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:49.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:49.486 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:49 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:49.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:49.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:49.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:49 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' 2024-04-26T21:22:50.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:50 smithi094 ceph-mon[31273]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:22:50.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:50 smithi094 ceph-mon[31273]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:22:50.277 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:50 smithi094 ceph-mon[31273]: pgmap v329: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:50 smithi027 ceph-mon[28085]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:22:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:50 smithi027 ceph-mon[28085]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:22:50.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:50 smithi027 ceph-mon[28085]: pgmap v329: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:50 smithi204 ceph-mon[32145]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-26T21:22:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:50 smithi204 ceph-mon[32145]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-26T21:22:50.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:50 smithi204 ceph-mon[32145]: pgmap v329: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 363, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 362, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1536372, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 226105, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 14, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-26T21:22:50.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.151088201, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.045797591, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.118027018, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.011381630, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 361, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.098960624, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000274129 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 132, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000067642, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000512 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1509296, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 145072 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 67, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 237, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 25050, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 450694, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 11, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.050463504, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.026789793, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.019518338, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000266567, 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:50.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 234, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.068779579, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000293929 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 95, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000044253, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000465 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 424512 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 436, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 629, 2024-04-26T21:22:50.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 156901, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 357954, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 9, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.192029835, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.104666306, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.078908172, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008562988, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 628, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.151617678, 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000241429 2024-04-26T21:22:50.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 992, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000503650, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000507 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 6080, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 11360 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-26T21:22:50.690 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 53, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002977280, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000056175 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000014652, 2024-04-26T21:22:50.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000007326 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 6, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 3, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001408052, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000469350 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 3, 2024-04-26T21:22:50.692 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 11, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 16, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 16, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 13, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 2, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 1, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 1, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 3, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 14, 2024-04-26T21:22:50.693 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 9, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 1, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 3, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:50.694 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:50.695 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 9, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 6, 2024-04-26T21:22:50.696 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 7, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:50.697 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 6, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 6, 2024-04-26T21:22:50.698 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4196308, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.022736854, 2024-04-26T21:22:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.004547370 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 20, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 21, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 1, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 19, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 21, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 2, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 16, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 18, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 2, 2024-04-26T21:22:50.700 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 1, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 1, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 51584, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 223608 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 6, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 6, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 21, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.701 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001408052, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000469350 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.703 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-26T21:22:50.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-26T21:22:50.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.707 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.708 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.709 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 3, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 2, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 2, 2024-04-26T21:22:50.713 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 610, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:50.714 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:50.715 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:50.716 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 256515, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 96, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 88, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 1, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 191, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 90533, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 342, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 42, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 12396, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 33, 2024-04-26T21:22:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 42, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 42, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.520991508, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.012404559 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 42, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 64 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 42, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-04-26T21:22:50.719 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 42, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 10, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 5, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 8, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 6, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-26T21:22:50.720 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 20, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-26T21:22:50.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-26T21:22:50.722 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 99, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 8, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 14, 2024-04-26T21:22:50.723 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 1 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:50.724 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 862, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1622715, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 862, 2024-04-26T21:22:50.725 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 862, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1622715, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:50.726 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 42, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 12396, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 42, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 12396, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:50.727 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 42, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 42, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 42, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 42, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:50.729 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x562c639e4500": { 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 6, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 2004, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:50.730 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 6, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2004, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-26T21:22:50.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-26T21:22:50.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-26T21:22:50.733 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T21:22:51.619 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-26T21:22:51.619 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-26T21:22:51.619 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-26T21:22:51.619 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=9 2024-04-26T21:22:51.620 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-26T21:22:51.620 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:51.620 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:51.620 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T21:22:51.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T21:22:52.082 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:52.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:52 smithi204 ceph-mon[32145]: pgmap v330: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:52.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:52 smithi027 ceph-mon[28085]: pgmap v330: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:52.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:52 smithi094 ceph-mon[31273]: pgmap v330: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:54.211 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:22:54.211 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2216, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2218, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 630476, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 704695, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.443545345, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.162699026, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.112371534, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.130108780, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2216, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.186074338, 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000083968 2024-04-26T21:22:54.212 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 255, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 239, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1503708, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 138858, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-04-26T21:22:54.213 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.108089535, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.035229013, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.085079945, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.011132839, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 239, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.032211613, 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000134776 2024-04-26T21:22:54.214 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1503152, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 138160 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3, 2024-04-26T21:22:54.218 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 169, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 588, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 424190, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.028749866, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.017312677, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.005524962, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000036973, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 166, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.026304399, 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000158460 2024-04-26T21:22:54.219 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 423872 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-26T21:22:54.220 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3782, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.190987485, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000050499 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.221 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:54.222 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-26T21:22:54.223 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:54.224 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:54.225 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-26T21:22:54.226 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:54.227 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 5, 2024-04-26T21:22:54.228 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4195917, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4195776, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.229 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 5 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-04-26T21:22:54.230 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49536, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.231 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-26T21:22:54.232 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.233 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.234 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-26T21:22:54.235 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.236 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.237 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.238 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.239 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.240 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-26T21:22:54.241 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.242 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-04-26T21:22:54.243 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3523, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:54.244 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:54.245 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:54.246 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 181343, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 47, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 191, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:54.247 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8512, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2219, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-04-26T21:22:54.248 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2219, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2219, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.862962520, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001290203 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2219, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2220 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2219, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2219, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-26T21:22:54.249 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1426, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 791, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-04-26T21:22:54.250 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-26T21:22:54.251 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 4, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 4, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-26T21:22:54.252 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:54.253 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2470, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1896817, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2470, 2024-04-26T21:22:54.254 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2470, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1896817, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:54.255 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2219, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1618, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2219, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1618, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:54.256 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2219, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2219, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2219, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2219, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:54.257 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.258 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55fedf80c780": { 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:54.259 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:54.260 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:54.260 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:54.260 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.260 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:54.260 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:22:54.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:54 smithi204 ceph-mon[32145]: pgmap v331: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:54.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:54 smithi027 ceph-mon[28085]: pgmap v331: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:54.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:54 smithi094 ceph-mon[31273]: pgmap v331: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-26T21:22:55.204 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T21:22:55.204 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T21:22:55.204 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-26T21:22:55.204 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-26T21:22:55.204 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-26T21:22:55.205 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:55.205 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:55.205 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T21:22:55.205 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T21:22:55.650 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:56.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:56 smithi204 ceph-mon[32145]: pgmap v332: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:56.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:56 smithi094 ceph-mon[31273]: pgmap v332: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:56.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:56 smithi027 ceph-mon[28085]: pgmap v332: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 794, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 796, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 210748, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 252556, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.176586162, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.061012136, 2024-04-26T21:22:57.654 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045466535, 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.045809038, 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 794, 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.083712875, 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000105431 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.655 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 897, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 883, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1659690, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 344425, 2024-04-26T21:22:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.254110221, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.082902341, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.117474834, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.059092472, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 882, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.084953179, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000096318 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.657 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1500752, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 138992 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 793, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 958, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 267696, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 642906, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:22:57.658 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.206549639, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.081751661, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.050838388, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.048425787, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 956, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.114860770, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000120147 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.659 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 392640 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3653, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.215827511, 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000059082 2024-04-26T21:22:57.660 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-26T21:22:57.661 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-26T21:22:57.662 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-26T21:22:57.663 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-04-26T21:22:57.664 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:22:57.665 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-26T21:22:57.666 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:22:57.667 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 7, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-26T21:22:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196449, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4196308, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.669 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 7 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-04-26T21:22:57.670 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49792, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-26T21:22:57.671 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.672 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.675 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.676 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-26T21:22:57.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.676 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-26T21:22:57.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.678 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.683 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 6124, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-26T21:22:57.684 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:22:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:22:57.686 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:22:57.687 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 190161, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 52, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8360, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 191, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:22:57.688 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8512, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2227, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-04-26T21:22:57.689 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2227, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2227, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.856962578, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001282874 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2227, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2228 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2227, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2227, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-26T21:22:57.690 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1431, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 794, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-26T21:22:57.691 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-26T21:22:57.692 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-26T21:22:57.693 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-26T21:22:57.694 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-26T21:22:57.695 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-26T21:22:57.696 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-26T21:22:57.696 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-26T21:22:57.696 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-26T21:22:57.696 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-26T21:22:57.697 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-26T21:22:57.698 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-26T21:22:57.699 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-26T21:22:57.700 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-26T21:22:57.701 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2480, 2024-04-26T21:22:57.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1899175, 2024-04-26T21:22:57.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:57.703 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2480, 2024-04-26T21:22:57.703 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:57.705 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2480, 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1899175, 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.706 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2227, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2150, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2227, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2150, 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:57.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:57.709 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-26T21:22:57.710 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:57.710 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:57.710 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:57.710 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2227, 2024-04-26T21:22:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2227, 2024-04-26T21:22:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2227, 2024-04-26T21:22:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2227, 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.712 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:57.713 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:57.714 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:57.714 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:57.714 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:57.714 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.715 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x558a737fc780": { 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:22:57.716 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:22:57.717 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:22:57.717 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:22:57.717 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:22:57.717 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:22:57.718 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:22:57.719 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.719 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:22:57.719 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:22:58.493 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:22:58 smithi204 ceph-mon[32145]: pgmap v333: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:58.563 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:22:58.564 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:22:58.564 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T21:22:58.564 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T21:22:58.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:22:58 smithi027 ceph-mon[28085]: pgmap v333: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:58.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:22:58 smithi094 ceph-mon[31273]: pgmap v333: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:22:59.002 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:23:00.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:00 smithi204 ceph-mon[32145]: pgmap v334: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:23:00.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:00 smithi027 ceph-mon[28085]: pgmap v334: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:23:00.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:00 smithi094 ceph-mon[31273]: pgmap v334: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:23:01.085 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:23:01.085 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:23:01.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1916, 2024-04-26T21:23:01.085 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1901, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3022437, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 667346, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.455302481, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.156265855, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.180762220, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.103382395, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1900, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.169567407, 2024-04-26T21:23:01.086 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000089246 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1500368, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 137872 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 10, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 173, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 4920, 2024-04-26T21:23:01.087 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 395242, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.033365289, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.017708222, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.009645622, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000244515, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 171, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.036504523, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000213476 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:01.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 393904 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 775, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 777, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 190756, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 246457, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-04-26T21:23:01.089 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.200577643, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.081678360, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045025536, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.055731593, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 775, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.075955167, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000098006 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3987, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.220476895, 2024-04-26T21:23:01.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000055298 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-26T21:23:01.092 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-26T21:23:01.093 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 30, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 30, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 30, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-26T21:23:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-04-26T21:23:01.095 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:23:01.096 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-26T21:23:01.097 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-26T21:23:01.098 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-26T21:23:01.099 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 195, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-26T21:23:01.100 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 5222039, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 5221927, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 195 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 21, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 24, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 3, 2024-04-26T21:23:01.101 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 11, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 11, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 30, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 30, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-26T21:23:01.102 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 50144, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.104 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.105 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.106 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-26T21:23:01.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.109 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-26T21:23:01.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.111 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-26T21:23:01.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.113 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.114 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 5, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-26T21:23:01.115 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 6126, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:23:01.116 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:23:01.117 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:23:01.118 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 189393, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 51, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8360, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 191, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-26T21:23:01.119 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 114996, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 525, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2444, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2444, 2024-04-26T21:23:01.120 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2444, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.976961017, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001218069 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2444, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2445 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2444, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2444, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-26T21:23:01.121 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1562, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 879, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-26T21:23:01.122 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-26T21:23:01.123 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-04-26T21:23:01.124 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-26T21:23:01.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2697, 2024-04-26T21:23:01.126 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2959682, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2697, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2697, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2959682, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:23:01.127 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2444, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1027745, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2444, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1027745, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.128 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2444, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2444, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2444, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2444, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.129 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:23:01.130 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55de97368780": { 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:01.131 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:01.132 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:23:01.917 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T21:23:01.917 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T21:23:01.918 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-26T21:23:01.918 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-26T21:23:01.918 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-26T21:23:01.918 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:23:01.918 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:23:01.918 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T21:23:01.918 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a125c556-040f-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T21:23:02.364 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:23:02.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:02 smithi204 ceph-mon[32145]: pgmap v335: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:02.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:02 smithi027 ceph-mon[28085]: pgmap v335: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:02.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:02 smithi094 ceph-mon[31273]: pgmap v335: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:04.380 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1598, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1598, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 481271, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 507341, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.363938274, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.146806959, 2024-04-26T21:23:04.381 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.088602613, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.089761070, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1597, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.145469197, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000091089 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 913, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 899, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1664438, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 351830, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-04-26T21:23:04.383 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.285184715, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.107780226, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.126773965, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.060110634, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 898, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.092792449, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000103332 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.384 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1502656, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 143472 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 176, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 588, 2024-04-26T21:23:04.385 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 396798, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.025370427, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016410010, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.003621899, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000042577, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 173, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.026720108, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000154451 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 396480 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-04-26T21:23:04.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3713, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.210278936, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000056633 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-04-26T21:23:04.388 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-04-26T21:23:04.390 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:23:04.391 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-04-26T21:23:04.392 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:23:04.393 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-04-26T21:23:04.394 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:23:04.395 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.396 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 6, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-04-26T21:23:04.397 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196308, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4196167, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 6 2024-04-26T21:23:04.398 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-04-26T21:23:04.399 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49536, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215416 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-04-26T21:23:04.400 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.401 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.402 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-04-26T21:23:04.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-04-26T21:23:04.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.406 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.411 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.412 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.413 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.414 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-04-26T21:23:04.415 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2812, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:23:04.416 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:23:04.417 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:23:04.418 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 190151, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 51, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8360, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 191, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-04-26T21:23:04.419 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8512, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2250, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2250, 2024-04-26T21:23:04.420 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2250, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.679964930, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001191095 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2250, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2251 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2250, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2250, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-04-26T21:23:04.421 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1446, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 802, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-04-26T21:23:04.422 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-04-26T21:23:04.423 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 99, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-04-26T21:23:04.424 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.425 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2510, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1904274, 2024-04-26T21:23:04.426 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2510, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2510, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1904274, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-04-26T21:23:04.427 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2250, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2009, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2250, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2009, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.428 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2250, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2250, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2250, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2250, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.429 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:04.430 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55ed786d6780": { 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-04-26T21:23:04.431 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-04-26T21:23:04.432 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-04-26T21:23:04.432 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:04 smithi204 ceph-mon[32145]: pgmap v336: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:04.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:04 smithi027 ceph-mon[28085]: pgmap v336: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:04.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:04 smithi094 ceph-mon[31273]: pgmap v336: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-26T21:23:05.308 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T21:23:05.309 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T21:23:05.320 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T21:23:05.320 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:23:05.320 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:23:05.388 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:23:05.388 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:23:05.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:23:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 -- 172.21.15.27:0/4010120985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f166010a950 msgr2=0x7f166010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 --2- 172.21.15.27:0/4010120985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f166010a950 0x7f166010ad30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1650009930 tx=0x7f165002a420 comp rx=0 tx=0).stop 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 -- 172.21.15.27:0/4010120985 shutdown_connections 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 --2- 172.21.15.27:0/4010120985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1660105300 0x7f1660111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 --2- 172.21.15.27:0/4010120985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660104940 0x7f1660104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 --2- 172.21.15.27:0/4010120985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f166010a950 0x7f166010ad30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 -- 172.21.15.27:0/4010120985 >> 172.21.15.27:0/4010120985 conn(0x7f16601003d0 msgr2=0x7f16601027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 -- 172.21.15.27:0/4010120985 shutdown_connections 2024-04-26T21:23:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.670+0000 7f1664e00640 1 -- 172.21.15.27:0/4010120985 wait complete. 2024-04-26T21:23:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.671+0000 7f1664e00640 1 Processor -- start 2024-04-26T21:23:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.671+0000 7f1664e00640 1 -- start start 2024-04-26T21:23:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660105300 0x7f166019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166010a950 0x7f166019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1660114ae0 con 0x7f1660104940 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1660114960 con 0x7f1660105300 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f1664e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1660114c60 con 0x7f166010a950 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f165f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f165f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46208/0 (socket says 172.21.15.27:46208) 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f165f400640 1 -- 172.21.15.27:0/3961517081 learned_addr learned my addr 172.21.15.27:0/3961517081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f165fe00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166010a950 0x7f166019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.672+0000 7f165ea00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660105300 0x7f166019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 -- 172.21.15.27:0/3961517081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166010a950 msgr2=0x7f166019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166010a950 0x7f166019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 -- 172.21.15.27:0/3961517081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660105300 msgr2=0x7f166019b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660105300 0x7f166019b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 -- 172.21.15.27:0/3961517081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16601a0010 con 0x7f1660104940 2024-04-26T21:23:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f165f400640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f165002a940 tx=0x7f1650003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165003f070 con 0x7f1660104940 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601a02a0 con 0x7f1660104940 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1650003e10 con 0x7f1660104940 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650035030 con 0x7f1660104940 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.673+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601a7b90 con 0x7f1660104940 2024-04-26T21:23:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.674+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660069670 con 0x7f1660104940 2024-04-26T21:23:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.677+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1650042070 con 0x7f1660104940 2024-04-26T21:23:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.678+0000 7f1657e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 0x7f16440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.678+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f16500f14b0 con 0x7f1660104940 2024-04-26T21:23:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.678+0000 7f165ea00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 0x7f16440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.679+0000 7f165ea00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 0x7f16440a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c005850 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.681+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16500bdaa0 con 0x7f1660104940 2024-04-26T21:23:05.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.886+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f166010ceb0 con 0x7f1660104940 2024-04-26T21:23:05.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.889+0000 7f1657e00640 1 -- 172.21.15.27:0/3961517081 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23197 (secure 0 0 0) 0x7f16500bd3a0 con 0x7f1660104940 2024-04-26T21:23:05.891 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:23:05.891 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:22:13.764482+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/542903082":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6800/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6801/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/2356002952":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/137865064":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/1196187505":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/238118775":"2024-04-27T21:12:12.895075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:23:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.891+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 msgr2=0x7f16440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 0x7f16440a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c005850 comp rx=0 tx=0).stop 2024-04-26T21:23:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 msgr2=0x7f166019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f165002a940 tx=0x7f1650003c20 comp rx=0 tx=0).stop 2024-04-26T21:23:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 shutdown_connections 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f16440a3d20 0x7f16440a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166010a950 0x7f166019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1660105300 0x7f166019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 --2- 172.21.15.27:0/3961517081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1660104940 0x7f166019b020 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 >> 172.21.15.27:0/3961517081 conn(0x7f16601003d0 msgr2=0x7f1660100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 shutdown_connections 2024-04-26T21:23:05.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:05.892+0000 7f1664e00640 1 -- 172.21.15.27:0/3961517081 wait complete. 2024-04-26T21:23:05.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:23:05.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:23:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.133+0000 7f5b3fe00640 1 -- 172.21.15.27:0/1218995734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 msgr2=0x7f5b4010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.133+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/1218995734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4010f250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009ca0 tx=0x7f5b3802a950 comp rx=0 tx=0).stop 2024-04-26T21:23:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 -- 172.21.15.27:0/1218995734 shutdown_connections 2024-04-26T21:23:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/1218995734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b401042b0 0x7f5b4010c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/1218995734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4010f250 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/1218995734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b40103990 0x7f5b40103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 -- 172.21.15.27:0/1218995734 >> 172.21.15.27:0/1218995734 conn(0x7f5b400fd1b0 msgr2=0x7f5b400ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 -- 172.21.15.27:0/1218995734 shutdown_connections 2024-04-26T21:23:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.134+0000 7f5b3fe00640 1 -- 172.21.15.27:0/1218995734 wait complete. 2024-04-26T21:23:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 Processor -- start 2024-04-26T21:23:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 -- start start 2024-04-26T21:23:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103990 0x7f5b4019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b401042b0 0x7f5b4019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40112910 con 0x7f5b4010cdb0 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40112790 con 0x7f5b401042b0 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.135+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40112a90 con 0x7f5b40103990 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46226/0 (socket says 172.21.15.27:46226) 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 -- 172.21.15.27:0/2312034710 learned_addr learned my addr 172.21.15.27:0/2312034710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3ea00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103990 0x7f5b4019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b36000640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b401042b0 0x7f5b4019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 -- 172.21.15.27:0/2312034710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103990 msgr2=0x7f5b4019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103990 0x7f5b4019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 -- 172.21.15.27:0/2312034710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b401042b0 msgr2=0x7f5b4019b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b401042b0 0x7f5b4019b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.136+0000 7f5b3f400640 1 -- 172.21.15.27:0/2312034710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b401a0020 con 0x7f5b4010cdb0 2024-04-26T21:23:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b3f400640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b3802ae70 tx=0x7f5b38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38034040 con 0x7f5b4010cdb0 2024-04-26T21:23:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b401a02b0 con 0x7f5b4010cdb0 2024-04-26T21:23:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b38030850 con 0x7f5b4010cdb0 2024-04-26T21:23:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004670 con 0x7f5b4010cdb0 2024-04-26T21:23:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.137+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401a7ba0 con 0x7f5b4010cdb0 2024-04-26T21:23:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.138+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b40069670 con 0x7f5b4010cdb0 2024-04-26T21:23:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.141+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5b38037020 con 0x7f5b4010cdb0 2024-04-26T21:23:06.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.143+0000 7f5b37e00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 0x7f5b240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.143+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f5b380f1790 con 0x7f5b4010cdb0 2024-04-26T21:23:06.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.143+0000 7f5b3ea00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 0x7f5b240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.144+0000 7f5b3ea00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 0x7f5b240a6260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b30005fd0 tx=0x7f5b30005850 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.148+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b380bdf20 con 0x7f5b4010cdb0 2024-04-26T21:23:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.357+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b401a7dd0 con 0x7f5b4010cdb0 2024-04-26T21:23:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.358+0000 7f5b37e00640 1 -- 172.21.15.27:0/2312034710 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f5b380bd820 con 0x7f5b4010cdb0 2024-04-26T21:23:06.359 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.360+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 msgr2=0x7f5b240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.360+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 0x7f5b240a6260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b30005fd0 tx=0x7f5b30005850 comp rx=0 tx=0).stop 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.360+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 msgr2=0x7f5b4019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.360+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b3802ae70 tx=0x7f5b38003f10 comp rx=0 tx=0).stop 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 shutdown_connections 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5b240a3da0 0x7f5b240a6260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4010cdb0 0x7f5b4019f8f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b401042b0 0x7f5b4019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 --2- 172.21.15.27:0/2312034710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b40103990 0x7f5b4019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 >> 172.21.15.27:0/2312034710 conn(0x7f5b400fd1b0 msgr2=0x7f5b401014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 shutdown_connections 2024-04-26T21:23:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.361+0000 7f5b3fe00640 1 -- 172.21.15.27:0/2312034710 wait complete. 2024-04-26T21:23:06.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:23:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 -- 172.21.15.27:0/44598623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181053a0 msgr2=0x7fec181118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 --2- 172.21.15.27:0/44598623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181053a0 0x7fec181118b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec10009cd0 tx=0x7fec1002a8f0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 -- 172.21.15.27:0/44598623 shutdown_connections 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 --2- 172.21.15.27:0/44598623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181053a0 0x7fec181118b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 --2- 172.21.15.27:0/44598623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec181049e0 0x7fec18104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 --2- 172.21.15.27:0/44598623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec1810a9f0 0x7fec1810add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 -- 172.21.15.27:0/44598623 >> 172.21.15.27:0/44598623 conn(0x7fec18100470 msgr2=0x7fec18102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 -- 172.21.15.27:0/44598623 shutdown_connections 2024-04-26T21:23:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.603+0000 7fec17200640 1 -- 172.21.15.27:0/44598623 wait complete. 2024-04-26T21:23:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 Processor -- start 2024-04-26T21:23:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 -- start start 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec181053a0 0x7fec1819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1810a9f0 0x7fec1819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec18114b80 con 0x7fec181049e0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec18114a00 con 0x7fec1810a9f0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.604+0000 7fec17200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fec18114d00 con 0x7fec181053a0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46242/0 (socket says 172.21.15.27:46242) 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 -- 172.21.15.27:0/2985344098 learned_addr learned my addr 172.21.15.27:0/2985344098 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15400640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec181053a0 0x7fec1819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec16800640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1810a9f0 0x7fec1819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 -- 172.21.15.27:0/2985344098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec181053a0 msgr2=0x7fec1819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec181053a0 0x7fec1819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 -- 172.21.15.27:0/2985344098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1810a9f0 msgr2=0x7fec1819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1810a9f0 0x7fec1819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.605+0000 7fec15e00640 1 -- 172.21.15.27:0/2985344098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec181a00f0 con 0x7fec181049e0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec15e00640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec0800d950 tx=0x7fec0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08017070 con 0x7fec181049e0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec08009c30 con 0x7fec181049e0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0800f620 con 0x7fec181049e0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec181a03e0 con 0x7fec181049e0 2024-04-26T21:23:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.606+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec181a7c70 con 0x7fec181049e0 2024-04-26T21:23:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.607+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec18069ac0 con 0x7fec181049e0 2024-04-26T21:23:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.609+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fec08010460 con 0x7fec181049e0 2024-04-26T21:23:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.611+0000 7fec0ea00640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 0x7febfc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.611+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fec080d0490 con 0x7fec181049e0 2024-04-26T21:23:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.611+0000 7fec15400640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 0x7febfc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.612+0000 7fec15400640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 0x7febfc0a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec18102b60 tx=0x7fec04005f90 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.616+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fec0809ca80 con 0x7fec181049e0 2024-04-26T21:23:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:06 smithi027 ceph-mon[28085]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:23:06.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:06 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3961517081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:06 smithi204 ceph-mon[32145]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:23:06.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:06 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3961517081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:06 smithi094 ceph-mon[31273]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-26T21:23:06.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:06 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3961517081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.826+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec1810cfd0 con 0x7fec181049e0 2024-04-26T21:23:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.827+0000 7fec0ea00640 1 -- 172.21.15.27:0/2985344098 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fec08009de0 con 0x7fec181049e0 2024-04-26T21:23:06.829 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:23:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 msgr2=0x7febfc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 0x7febfc0a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec18102b60 tx=0x7fec04005f90 comp rx=0 tx=0).stop 2024-04-26T21:23:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 msgr2=0x7fec1819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec0800d950 tx=0x7fec0800de10 comp rx=0 tx=0).stop 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 shutdown_connections 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7febfc0a3da0 0x7febfc0a6260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec181053a0 0x7fec1819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1810a9f0 0x7fec1819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 --2- 172.21.15.27:0/2985344098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec181049e0 0x7fec1819b100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 >> 172.21.15.27:0/2985344098 conn(0x7fec18100470 msgr2=0x7fec18100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 shutdown_connections 2024-04-26T21:23:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:06.830+0000 7fec17200640 1 -- 172.21.15.27:0/2985344098 wait complete. 2024-04-26T21:23:06.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 -- 172.21.15.27:0/711962658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8105490 msgr2=0x7f1ec81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 --2- 172.21.15.27:0/711962658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8105490 0x7f1ec81119a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009cd0 tx=0x7f1ec002a8f0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 -- 172.21.15.27:0/711962658 shutdown_connections 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 --2- 172.21.15.27:0/711962658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8105490 0x7f1ec81119a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 --2- 172.21.15.27:0/711962658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec8104ad0 0x7f1ec8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 --2- 172.21.15.27:0/711962658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec810ab70 0x7f1ec810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 -- 172.21.15.27:0/711962658 >> 172.21.15.27:0/711962658 conn(0x7f1ec8100660 msgr2=0x7f1ec8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 -- 172.21.15.27:0/711962658 shutdown_connections 2024-04-26T21:23:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.063+0000 7f1ecca00640 1 -- 172.21.15.27:0/711962658 wait complete. 2024-04-26T21:23:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.064+0000 7f1ecca00640 1 Processor -- start 2024-04-26T21:23:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.064+0000 7f1ecca00640 1 -- start start 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec8105490 0x7f1ec819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 0x7f1ec819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8114ca0 con 0x7f1ec8104ad0 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8114b20 con 0x7f1ec810ab70 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ecca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8114e20 con 0x7f1ec8105490 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46260/0 (socket says 172.21.15.27:46260) 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ec5e00640 1 -- 172.21.15.27:0/1789534479 learned_addr learned my addr 172.21.15.27:0/1789534479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ebfe00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec8105490 0x7f1ec819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.065+0000 7f1ec6800640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 0x7f1ec819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 -- 172.21.15.27:0/1789534479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec8105490 msgr2=0x7f1ec819b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec8105490 0x7f1ec819b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 -- 172.21.15.27:0/1789534479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 msgr2=0x7f1ec819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 0x7f1ec819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 -- 172.21.15.27:0/1789534479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a0210 con 0x7f1ec8104ad0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec6800640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 0x7f1ec819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ec5e00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800d770 tx=0x7f1eb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8017070 con 0x7f1ec8104ad0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a0500 con 0x7f1ec8104ad0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1eb80054b0 con 0x7f1ec8104ad0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004c60 con 0x7f1ec8104ad0 2024-04-26T21:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.066+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a7d90 con 0x7f1ec8104ad0 2024-04-26T21:23:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.068+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8069ba0 con 0x7f1ec8104ad0 2024-04-26T21:23:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.070+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1eb8004e00 con 0x7f1ec8104ad0 2024-04-26T21:23:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.071+0000 7f1ebea00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 0x7f1eac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.071+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f1eb80d06b0 con 0x7f1ec8104ad0 2024-04-26T21:23:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.072+0000 7f1ebfe00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 0x7f1eac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.072+0000 7f1ebfe00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 0x7f1eac0a6260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8102d00 tx=0x7f1eb000b590 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.074+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1eb809cd20 con 0x7f1ec8104ad0 2024-04-26T21:23:07.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.281+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ec802dad0 con 0x7f1ec8104ad0 2024-04-26T21:23:07.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.282+0000 7f1ebea00640 1 -- 172.21.15.27:0/1789534479 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f1eb809c620 con 0x7f1ec8104ad0 2024-04-26T21:23:07.283 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 msgr2=0x7f1eac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 0x7f1eac0a6260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8102d00 tx=0x7f1eb000b590 comp rx=0 tx=0).stop 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 msgr2=0x7f1ec819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800d770 tx=0x7f1eb800dc30 comp rx=0 tx=0).stop 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 shutdown_connections 2024-04-26T21:23:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1eac0a3da0 0x7f1eac0a6260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ec8105490 0x7f1ec819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec810ab70 0x7f1ec819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 --2- 172.21.15.27:0/1789534479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8104ad0 0x7f1ec819b220 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 >> 172.21.15.27:0/1789534479 conn(0x7f1ec8100660 msgr2=0x7f1ec8100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 shutdown_connections 2024-04-26T21:23:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.285+0000 7f1ecca00640 1 -- 172.21.15.27:0/1789534479 wait complete. 2024-04-26T21:23:07.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:23:07.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 -- 172.21.15.27:0/2864475550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa218108fa0 msgr2=0x7fa218109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 --2- 172.21.15.27:0/2864475550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa218108fa0 0x7fa218109380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa208009900 tx=0x7fa20802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 -- 172.21.15.27:0/2864475550 shutdown_connections 2024-04-26T21:23:07.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 --2- 172.21.15.27:0/2864475550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21810e060 0x7fa218115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 --2- 172.21.15.27:0/2864475550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa218109950 0x7fa21810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 --2- 172.21.15.27:0/2864475550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa218108fa0 0x7fa218109380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 -- 172.21.15.27:0/2864475550 >> 172.21.15.27:0/2864475550 conn(0x7fa2180784f0 msgr2=0x7fa21807a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 -- 172.21.15.27:0/2864475550 shutdown_connections 2024-04-26T21:23:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.521+0000 7fa21d600640 1 -- 172.21.15.27:0/2864475550 wait complete. 2024-04-26T21:23:07.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 Processor -- start 2024-04-26T21:23:07.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 -- start start 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa218108fa0 0x7fa21819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa21810e060 0x7fa21819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa218118f20 con 0x7fa2181a3ca0 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa218118da0 con 0x7fa21810e060 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.522+0000 7fa21d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2181190a0 con 0x7fa218108fa0 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46272/0 (socket says 172.21.15.27:46272) 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 -- 172.21.15.27:0/3459014293 learned_addr learned my addr 172.21.15.27:0/3459014293 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa217e00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa218108fa0 0x7fa21819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa217400640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa21810e060 0x7fa21819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 -- 172.21.15.27:0/3459014293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa218108fa0 msgr2=0x7fa21819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa218108fa0 0x7fa21819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 -- 172.21.15.27:0/3459014293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa21810e060 msgr2=0x7fa21819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa21810e060 0x7fa21819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 -- 172.21.15.27:0/3459014293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2181a4790 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.523+0000 7fa21cc00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b9e0 tx=0x7fa21000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.524+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210019070 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.524+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa210014440 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.524+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100075b0 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.524+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2181a4a80 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.524+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2181ac220 con 0x7fa2181a3ca0 2024-04-26T21:23:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.525+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2181a0730 con 0x7fa2181a3ca0 2024-04-26T21:23:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.527+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa21001c070 con 0x7fa2181a3ca0 2024-04-26T21:23:07.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.529+0000 7fa214c00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 0x7fa1fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:07.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.529+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fa2100cfd50 con 0x7fa2181a3ca0 2024-04-26T21:23:07.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.529+0000 7fa217e00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 0x7fa1fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.530+0000 7fa217e00640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 0x7fa1fc0a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa208006010 tx=0x7fa208005a20 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:07.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.533+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa21009c3b0 con 0x7fa2181a3ca0 2024-04-26T21:23:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:07 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2312034710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:07 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2985344098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:07 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1789534479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:07 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2312034710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:07 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2985344098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:07 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1789534479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.759+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa218069aa0 con 0x7fa2181a3ca0 2024-04-26T21:23:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.760+0000 7fa214c00640 1 -- 172.21.15.27:0/3459014293 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v99) ==== 81+0+999 (secure 0 0 0) 0x7fa21009bcb0 con 0x7fa2181a3ca0 2024-04-26T21:23:07.761 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/128242469 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2522153056 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3447033143 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2094082344 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/128242469 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2239698495 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2094082344 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1042211156 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2901658935 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2516445598 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542903082 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3365757774 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3365757774 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.763 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2356002952 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.763 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/137865064 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.763 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1196187505 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.763 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/238118775 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:listed 17 entries 2024-04-26T21:23:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 msgr2=0x7fa1fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 0x7fa1fc0a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa208006010 tx=0x7fa208005a20 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 msgr2=0x7fa2181a4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b9e0 tx=0x7fa21000bea0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 shutdown_connections 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fa1fc0a3da0 0x7fa1fc0a6260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2181a3ca0 0x7fa2181a4150 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa21810e060 0x7fa21819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 --2- 172.21.15.27:0/3459014293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa218108fa0 0x7fa21819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 >> 172.21.15.27:0/3459014293 conn(0x7fa2180784f0 msgr2=0x7fa21807a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 shutdown_connections 2024-04-26T21:23:07.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:07.763+0000 7fa21d600640 1 -- 172.21.15.27:0/3459014293 wait complete. 2024-04-26T21:23:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:07 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2312034710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:07 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2985344098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:07 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1789534479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:07.784 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-26T21:23:07.784 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:23:07.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:23:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 -- 172.21.15.27:0/538593602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca810e060 msgr2=0x7f1ca8115c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/538593602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca810e060 0x7f1ca8115c30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 -- 172.21.15.27:0/538593602 shutdown_connections 2024-04-26T21:23:08.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/538593602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca810e060 0x7f1ca8115c30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/538593602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca8109950 0x7f1ca810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/538593602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca8108fa0 0x7f1ca8109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 -- 172.21.15.27:0/538593602 >> 172.21.15.27:0/538593602 conn(0x7f1ca80784b0 msgr2=0x7f1ca807a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 -- 172.21.15.27:0/538593602 shutdown_connections 2024-04-26T21:23:08.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.056+0000 7f1ca7e00640 1 -- 172.21.15.27:0/538593602 wait complete. 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.057+0000 7f1ca7e00640 1 Processor -- start 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 -- start start 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 0x7f1ca819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca81a3cb0 0x7f1ca81a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8118ed0 con 0x7f1ca8108fa0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8118d50 con 0x7f1ca81a3cb0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8119050 con 0x7f1ca810e060 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46280/0 (socket says 172.21.15.27:46280) 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca6a00640 1 -- 172.21.15.27:0/3420767467 learned_addr learned my addr 172.21.15.27:0/3420767467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.058+0000 7f1ca7400640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca81a3cb0 0x7f1ca81a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1c9e000640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 0x7f1ca819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 -- 172.21.15.27:0/3420767467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 msgr2=0x7f1ca819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 0x7f1ca819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 -- 172.21.15.27:0/3420767467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca81a3cb0 msgr2=0x7f1ca81a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca81a3cb0 0x7f1ca81a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 -- 172.21.15.27:0/3420767467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca81a47a0 con 0x7f1ca8108fa0 2024-04-26T21:23:08.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1c9e000640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 0x7f1ca819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca6a00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800ea80 tx=0x7f1c9800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800ce40 con 0x7f1ca8108fa0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c98005020 con 0x7f1ca8108fa0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980116d0 con 0x7f1ca8108fa0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.059+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca81a4a90 con 0x7f1ca8108fa0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.060+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81ac1d0 con 0x7f1ca8108fa0 2024-04-26T21:23:08.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.061+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca8069ac0 con 0x7f1ca8108fa0 2024-04-26T21:23:08.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.063+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1c98004a40 con 0x7f1ca8108fa0 2024-04-26T21:23:08.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.064+0000 7f1c9fe00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 0x7f1c8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.064+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f1c980d0e50 con 0x7f1ca8108fa0 2024-04-26T21:23:08.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.065+0000 7f1c9e000640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 0x7f1c8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.065+0000 7f1c9e000640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 0x7f1c8c0a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005fd0 tx=0x7f1c900094d0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.067+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1c98016030 con 0x7f1ca8108fa0 2024-04-26T21:23:08.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.266+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ca81a0740 con 0x7f1ca8108fa0 2024-04-26T21:23:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.269+0000 7f1c9fe00640 1 -- 172.21.15.27:0/3420767467 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23197 (secure 0 0 0) 0x7f1c9809d520 con 0x7f1ca8108fa0 2024-04-26T21:23:08.271 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:23:08.271 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:22:13.764482+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/542903082":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6800/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6801/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/2356002952":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/137865064":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/1196187505":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/238118775":"2024-04-27T21:12:12.895075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.271+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 msgr2=0x7f1c8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.271+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 0x7f1c8c0a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005fd0 tx=0x7f1c900094d0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.271+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 msgr2=0x7f1ca819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.271+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800ea80 tx=0x7f1c9800ef40 comp rx=0 tx=0).stop 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 shutdown_connections 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f1c8c0a3da0 0x7f1c8c0a6260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ca810e060 0x7f1ca819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca81a3cb0 0x7f1ca81a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 --2- 172.21.15.27:0/3420767467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ca8108fa0 0x7f1ca819f3f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 >> 172.21.15.27:0/3420767467 conn(0x7f1ca80784b0 msgr2=0x7f1ca807a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 shutdown_connections 2024-04-26T21:23:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.272+0000 7f1ca7e00640 1 -- 172.21.15.27:0/3420767467 wait complete. 2024-04-26T21:23:08.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:03:22.088553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:38.486492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:39.830620+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:23:08.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:23:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 -- 172.21.15.27:0/2026998210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 msgr2=0x7ff43c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 --2- 172.21.15.27:0/2026998210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c10f2d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009900 tx=0x7ff42c02a480 comp rx=0 tx=0).stop 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 -- 172.21.15.27:0/2026998210 shutdown_connections 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 --2- 172.21.15.27:0/2026998210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 0x7ff43c115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 --2- 172.21.15.27:0/2026998210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 0x7ff43c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 --2- 172.21.15.27:0/2026998210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c10f2d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 -- 172.21.15.27:0/2026998210 >> 172.21.15.27:0/2026998210 conn(0x7ff43c078750 msgr2=0x7ff43c107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 -- 172.21.15.27:0/2026998210 shutdown_connections 2024-04-26T21:23:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.513+0000 7ff440c00640 1 -- 172.21.15.27:0/2026998210 wait complete. 2024-04-26T21:23:08.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.514+0000 7ff440c00640 1 Processor -- start 2024-04-26T21:23:08.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.514+0000 7ff440c00640 1 -- start start 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.514+0000 7ff440c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 0x7ff43c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.514+0000 7ff440c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 0x7ff43c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.514+0000 7ff440c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff440c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c118f50 con 0x7ff43c10eef0 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff440c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c118dd0 con 0x7ff43c108ed0 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff440c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1190d0 con 0x7ff43c109870 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46298/0 (socket says 172.21.15.27:46298) 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 -- 172.21.15.27:0/95766991 learned_addr learned my addr 172.21.15.27:0/95766991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43b400640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 0x7ff43c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43aa00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 0x7ff43c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 -- 172.21.15.27:0/95766991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 msgr2=0x7ff43c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 0x7ff43c19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 -- 172.21.15.27:0/95766991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 msgr2=0x7ff43c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 0x7ff43c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.515+0000 7ff43be00640 1 -- 172.21.15.27:0/95766991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c1a4480 con 0x7ff43c10eef0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff43be00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff43400e960 tx=0x7ff43400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434004d20 con 0x7ff43c10eef0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff434005020 con 0x7ff43c10eef0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340116d0 con 0x7ff43c10eef0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1a4770 con 0x7ff43c10eef0 2024-04-26T21:23:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.516+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1ac000 con 0x7ff43c10eef0 2024-04-26T21:23:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.517+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c069ac0 con 0x7ff43c10eef0 2024-04-26T21:23:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.519+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff4340051d0 con 0x7ff43c10eef0 2024-04-26T21:23:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.521+0000 7ff433e00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.521+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7ff4340d0d10 con 0x7ff43c10eef0 2024-04-26T21:23:08.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.521+0000 7ff43b400640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.522+0000 7ff43b400640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 0x7ff4200a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff42c000c00 tx=0x7ff42c005a20 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.525+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4340d6020 con 0x7ff43c10eef0 2024-04-26T21:23:08.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:08 smithi094 ceph-mon[31273]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-26T21:23:08.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:08 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3459014293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:08.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:08 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3420767467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:08.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.733+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff43c110d50 con 0x7ff43c10eef0 2024-04-26T21:23:08.734 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:08 smithi027 ceph-mon[28085]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-26T21:23:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:08 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3459014293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:08.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:08 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3420767467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:08.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.734+0000 7ff433e00640 1 -- 172.21.15.27:0/95766991 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7ff434016030 con 0x7ff43c10eef0 2024-04-26T21:23:08.735 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-26T21:23:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.736+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 msgr2=0x7ff4200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.736+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 0x7ff4200a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff42c000c00 tx=0x7ff42c005a20 comp rx=0 tx=0).stop 2024-04-26T21:23:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.736+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 msgr2=0x7ff43c1a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.736+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff43400e960 tx=0x7ff43400ee20 comp rx=0 tx=0).stop 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 shutdown_connections 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff43c109870 0x7ff43c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff43c10eef0 0x7ff43c1a3d50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 --2- 172.21.15.27:0/95766991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff43c108ed0 0x7ff43c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 >> 172.21.15.27:0/95766991 conn(0x7ff43c078750 msgr2=0x7ff43c0799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 shutdown_connections 2024-04-26T21:23:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.737+0000 7ff440c00640 1 -- 172.21.15.27:0/95766991 wait complete. 2024-04-26T21:23:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:08 smithi204 ceph-mon[32145]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-26T21:23:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:08 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3459014293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:08.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:08 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3420767467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:08.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 -- 172.21.15.27:0/2430079361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3810f210 msgr2=0x7f4c38111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 --2- 172.21.15.27:0/2430079361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3810f210 0x7f4c38111600 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009900 tx=0x7f4c2802a480 comp rx=0 tx=0).stop 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 -- 172.21.15.27:0/2430079361 shutdown_connections 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 --2- 172.21.15.27:0/2430079361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3810f210 0x7f4c38111600 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 --2- 172.21.15.27:0/2430079361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c38075520 0x7f4c3810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 --2- 172.21.15.27:0/2430079361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 0x7f4c38074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 -- 172.21.15.27:0/2430079361 >> 172.21.15.27:0/2430079361 conn(0x7f4c38079d80 msgr2=0x7f4c3807a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 -- 172.21.15.27:0/2430079361 shutdown_connections 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.982+0000 7f4c3f000640 1 -- 172.21.15.27:0/2430079361 wait complete. 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.983+0000 7f4c3f000640 1 Processor -- start 2024-04-26T21:23:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 -- start start 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 0x7f4c3819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 0x7f4c3819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38114d20 con 0x7f4c38075520 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38114ba0 con 0x7f4c3810f210 2024-04-26T21:23:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c3f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38114ea0 con 0x7f4c38074c00 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c37400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49312/0 (socket says 172.21.15.27:49312) 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.984+0000 7f4c37400640 1 -- 172.21.15.27:0/4234520455 learned_addr learned my addr 172.21.15.27:0/4234520455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37e00640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 0x7f4c3819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c3d000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 0x7f4c3819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 -- 172.21.15.27:0/4234520455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 msgr2=0x7f4c3819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 0x7f4c3819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 -- 172.21.15.27:0/4234520455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 msgr2=0x7f4c3819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 0x7f4c3819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 -- 172.21.15.27:0/4234520455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381a02f0 con 0x7f4c38075520 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c3d000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 0x7f4c3819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.985+0000 7f4c37400640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000ba50 tx=0x7f4c2000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.986+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20019070 con 0x7f4c38075520 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.986+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c2000cbe0 con 0x7f4c38075520 2024-04-26T21:23:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.986+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c200075d0 con 0x7f4c38075520 2024-04-26T21:23:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.986+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381a05e0 con 0x7f4c38075520 2024-04-26T21:23:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.986+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a7e70 con 0x7f4c38075520 2024-04-26T21:23:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.987+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c38069ba0 con 0x7f4c38075520 2024-04-26T21:23:08.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.989+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4c2001c070 con 0x7f4c38075520 2024-04-26T21:23:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.990+0000 7f4c34c00640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 0x7f4c1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.990+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7f4c200cfe80 con 0x7f4c38075520 2024-04-26T21:23:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.991+0000 7f4c37e00640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 0x7f4c1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.991+0000 7f4c37e00640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 0x7f4c1c0a6260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005fd0 tx=0x7f4c30005e20 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:08.995+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4c2009c5e0 con 0x7f4c38075520 2024-04-26T21:23:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.212+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c38109e00 con 0x7f4c38075520 2024-04-26T21:23:09.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.213+0000 7f4c34c00640 1 -- 172.21.15.27:0/4234520455 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f4c2009bee0 con 0x7f4c38075520 2024-04-26T21:23:09.215 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-26T21:23:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 msgr2=0x7f4c1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 0x7f4c1c0a6260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005fd0 tx=0x7f4c30005e20 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 msgr2=0x7f4c3819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000ba50 tx=0x7f4c2000bf10 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 shutdown_connections 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f4c1c0a3da0 0x7f4c1c0a6260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c3810f210 0x7f4c3819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c38075520 0x7f4c3819b840 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 --2- 172.21.15.27:0/4234520455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c38074c00 0x7f4c3819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.216+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 >> 172.21.15.27:0/4234520455 conn(0x7f4c38079d80 msgr2=0x7f4c380ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.217+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 shutdown_connections 2024-04-26T21:23:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.217+0000 7f4c3f000640 1 -- 172.21.15.27:0/4234520455 wait complete. 2024-04-26T21:23:09.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:23:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 -- 172.21.15.27:0/1645783297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101490 msgr2=0x7ff4f4101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1645783297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101490 0x7ff4f4101870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009900 tx=0x7ff4e802a410 comp rx=0 tx=0).stop 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 -- 172.21.15.27:0/1645783297 shutdown_connections 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1645783297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f4101e40 0x7ff4f410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1645783297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f410f040 0x7ff4f4111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1645783297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101490 0x7ff4f4101870 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 -- 172.21.15.27:0/1645783297 >> 172.21.15.27:0/1645783297 conn(0x7ff4f4078410 msgr2=0x7ff4f40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 -- 172.21.15.27:0/1645783297 shutdown_connections 2024-04-26T21:23:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.461+0000 7ff4fa600640 1 -- 172.21.15.27:0/1645783297 wait complete. 2024-04-26T21:23:09.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.462+0000 7ff4fa600640 1 Processor -- start 2024-04-26T21:23:09.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.462+0000 7ff4fa600640 1 -- start start 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4101490 0x7ff4f419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f410f040 0x7ff4f419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4114aa0 con 0x7ff4f4101e40 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4114920 con 0x7ff4f4101490 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4fa600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4114c20 con 0x7ff4f410f040 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49320/0 (socket says 172.21.15.27:49320) 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4f3e00640 1 -- 172.21.15.27:0/1893389989 learned_addr learned my addr 172.21.15.27:0/1893389989 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.463+0000 7ff4f9200640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4101490 0x7ff4f419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f9c00640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f410f040 0x7ff4f419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 -- 172.21.15.27:0/1893389989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f410f040 msgr2=0x7ff4f419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f410f040 0x7ff4f419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 -- 172.21.15.27:0/1893389989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4101490 msgr2=0x7ff4f419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4101490 0x7ff4f419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 -- 172.21.15.27:0/1893389989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a0050 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f3e00640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d950 tx=0x7ff4e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0017070 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4e0009c30 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000f620 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.464+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a0340 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.465+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a7bd0 con 0x7ff4f4101e40 2024-04-26T21:23:09.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.466+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f4069670 con 0x7ff4f4101e40 2024-04-26T21:23:09.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.468+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff4e0010460 con 0x7ff4f4101e40 2024-04-26T21:23:09.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.469+0000 7ff4f1600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 0x7ff4dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.470+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7ff4e00d0500 con 0x7ff4f4101e40 2024-04-26T21:23:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.470+0000 7ff4f9200640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 0x7ff4dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.471+0000 7ff4f9200640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 0x7ff4dc0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006010 tx=0x7ff4e8005a20 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:09.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.474+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4e009caf0 con 0x7ff4f4101e40 2024-04-26T21:23:09.668 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/95766991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.668 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:09 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4234520455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.685+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4f4109c40 con 0x7ff4f4101e40 2024-04-26T21:23:09.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.685+0000 7ff4f1600640 1 -- 172.21.15.27:0/1893389989 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7ff4e0009de0 con 0x7ff4f4101e40 2024-04-26T21:23:09.687 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-26T21:23:09.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 msgr2=0x7ff4dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 0x7ff4dc0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006010 tx=0x7ff4e8005a20 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 msgr2=0x7ff4f419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d950 tx=0x7ff4e000de10 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 shutdown_connections 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff4dc0a3da0 0x7ff4dc0a6260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4f410f040 0x7ff4f419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f4101e40 0x7ff4f419b5f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 --2- 172.21.15.27:0/1893389989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f4101490 0x7ff4f419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.688+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 >> 172.21.15.27:0/1893389989 conn(0x7ff4f4078410 msgr2=0x7ff4f4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.689+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 shutdown_connections 2024-04-26T21:23:09.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.689+0000 7ff4fa600640 1 -- 172.21.15.27:0/1893389989 wait complete. 2024-04-26T21:23:09.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:23:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:09 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/95766991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:09 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4234520455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:09 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/95766991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:09 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4234520455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 -- 172.21.15.27:0/3450349298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce1810a920 msgr2=0x7fce1810ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 --2- 172.21.15.27:0/3450349298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce1810a920 0x7fce1810ad00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce08009930 tx=0x7fce0802a440 comp rx=0 tx=0).stop 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 -- 172.21.15.27:0/3450349298 shutdown_connections 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 --2- 172.21.15.27:0/3450349298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce181052d0 0x7fce181117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 --2- 172.21.15.27:0/3450349298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18104910 0x7fce18104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 --2- 172.21.15.27:0/3450349298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce1810a920 0x7fce1810ad00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 -- 172.21.15.27:0/3450349298 >> 172.21.15.27:0/3450349298 conn(0x7fce181003a0 msgr2=0x7fce181027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 -- 172.21.15.27:0/3450349298 shutdown_connections 2024-04-26T21:23:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.922+0000 7fce17e00640 1 -- 172.21.15.27:0/3450349298 wait complete. 2024-04-26T21:23:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 Processor -- start 2024-04-26T21:23:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 -- start start 2024-04-26T21:23:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce181052d0 0x7fce1819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 0x7fce1819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce18114ab0 con 0x7fce18104910 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce18114930 con 0x7fce181052d0 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.923+0000 7fce17e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce18114c30 con 0x7fce1810a920 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49336/0 (socket says 172.21.15.27:49336) 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 -- 172.21.15.27:0/449405115 learned_addr learned my addr 172.21.15.27:0/449405115 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16000640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce181052d0 0x7fce1819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce17400640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 0x7fce1819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 -- 172.21.15.27:0/449405115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 msgr2=0x7fce1819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 0x7fce1819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 -- 172.21.15.27:0/449405115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce181052d0 msgr2=0x7fce1819b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce181052d0 0x7fce1819b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce16a00640 1 -- 172.21.15.27:0/449405115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1819ffd0 con 0x7fce18104910 2024-04-26T21:23:09.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.924+0000 7fce17400640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 0x7fce1819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce16a00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce0802a960 tx=0x7fce08003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0803f070 con 0x7fce18104910 2024-04-26T21:23:09.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce0803bdc0 con 0x7fce18104910 2024-04-26T21:23:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08035030 con 0x7fce18104910 2024-04-26T21:23:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181a0260 con 0x7fce18104910 2024-04-26T21:23:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.925+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181a7b50 con 0x7fce18104910 2024-04-26T21:23:09.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.927+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18069ac0 con 0x7fce18104910 2024-04-26T21:23:09.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.929+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fce08042070 con 0x7fce18104910 2024-04-26T21:23:09.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.930+0000 7fce0f400640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 0x7fcdfc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:09.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.930+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8593+0+0 (secure 0 0 0) 0x7fce080f1210 con 0x7fce18104910 2024-04-26T21:23:09.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.931+0000 7fce16000640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 0x7fcdfc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:09.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.931+0000 7fce16000640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 0x7fcdfc0a6260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce04005e10 tx=0x7fce04005da0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:09.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:09.935+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fce080bd860 con 0x7fce18104910 2024-04-26T21:23:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.162+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce1819c330 con 0x7fce18104910 2024-04-26T21:23:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.163+0000 7fce0f400640 1 -- 172.21.15.27:0/449405115 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v99) ==== 81+0+999 (secure 0 0 0) 0x7fce080bd160 con 0x7fce18104910 2024-04-26T21:23:10.164 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/128242469 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:10.164 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2522153056 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3447033143 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2094082344 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/128242469 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2239698495 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2094082344 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1042211156 2024-04-27T20:59:25.698004+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2901658935 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2516445598 2024-04-27T21:00:01.870639+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/542903082 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3365757774 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3365757774 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2356002952 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/137865064 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.165 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1196187505 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.166 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/238118775 2024-04-27T21:12:12.895075+0000 2024-04-26T21:23:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:listed 17 entries 2024-04-26T21:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 msgr2=0x7fcdfc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 0x7fcdfc0a6260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce04005e10 tx=0x7fce04005da0 comp rx=0 tx=0).stop 2024-04-26T21:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 msgr2=0x7fce1819b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce0802a960 tx=0x7fce08003d60 comp rx=0 tx=0).stop 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 shutdown_connections 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fcdfc0a3da0 0x7fcdfc0a6260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce1810a920 0x7fce1819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce181052d0 0x7fce1819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 --2- 172.21.15.27:0/449405115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce18104910 0x7fce1819b030 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 >> 172.21.15.27:0/449405115 conn(0x7fce181003a0 msgr2=0x7fce18100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 shutdown_connections 2024-04-26T21:23:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:10.166+0000 7fce17e00640 1 -- 172.21.15.27:0/449405115 wait complete. 2024-04-26T21:23:10.186 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-04-26T21:23:10.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:23:10.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:23:10.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[80721]: fuse finished with error 0 and tester_r 0 2024-04-26T21:23:10.269 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:23:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:10 smithi027 ceph-mon[28085]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:23:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:10 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1893389989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:10.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:10 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/449405115' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:10 smithi204 ceph-mon[32145]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:23:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:10 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1893389989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:10.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:10 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/449405115' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:10 smithi094 ceph-mon[31273]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T21:23:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:10 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1893389989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:23:10.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:10 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/449405115' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:23:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:12 smithi027 ceph-mon[28085]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:23:12.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:12 smithi027 ceph-mon[28085]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T21:23:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:12 smithi204 ceph-mon[32145]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:23:12.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:12 smithi204 ceph-mon[32145]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T21:23:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:12 smithi094 ceph-mon[31273]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T21:23:12.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:12 smithi094 ceph-mon[31273]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T21:23:14.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:14 smithi027 ceph-mon[28085]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:23:14.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:14 smithi204 ceph-mon[32145]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:23:14.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:14 smithi094 ceph-mon[31273]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-26T21:23:16.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:23:16.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:23:16.372 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:23:16.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:23:16.452 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-26T21:23:16.512 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:23:16.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:23:16.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:23:16.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:23:16.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-26T21:23:16.666 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:16 smithi027 ceph-mon[28085]: pgmap v343: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:16.708 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T21:23:16.719 INFO:tasks.cephadm:Teardown begin 2024-04-26T21:23:16.719 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:23:16.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:23:16.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:16 smithi204 ceph-mon[32145]: pgmap v343: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:16.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:16 smithi094 ceph-mon[31273]: pgmap v343: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 -- 172.21.15.27:0/1824699664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109bc0 msgr2=0x7ff63c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 --2- 172.21.15.27:0/1824699664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109bc0 0x7ff63c10dc00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff624009900 tx=0x7ff62402a400 comp rx=0 tx=0).stop 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 -- 172.21.15.27:0/1824699664 shutdown_connections 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 --2- 172.21.15.27:0/1824699664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c10e2e0 0x7ff63c115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 --2- 172.21.15.27:0/1824699664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109bc0 0x7ff63c10dc00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 --2- 172.21.15.27:0/1824699664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c109210 0x7ff63c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.912+0000 7ff642000640 1 -- 172.21.15.27:0/1824699664 >> 172.21.15.27:0/1824699664 conn(0x7ff63c076f50 msgr2=0x7ff63c079370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.913+0000 7ff642000640 1 -- 172.21.15.27:0/1824699664 shutdown_connections 2024-04-26T21:23:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.913+0000 7ff642000640 1 -- 172.21.15.27:0/1824699664 wait complete. 2024-04-26T21:23:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.913+0000 7ff642000640 1 Processor -- start 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 -- start start 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c10e2e0 0x7ff63c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c1a3f10 0x7ff63c1a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c119100 con 0x7ff63c109210 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c118f80 con 0x7ff63c1a3f10 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff642000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c119280 con 0x7ff63c10e2e0 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff63b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff63b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49348/0 (socket says 172.21.15.27:49348) 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.914+0000 7ff63b400640 1 -- 172.21.15.27:0/2082706214 learned_addr learned my addr 172.21.15.27:0/2082706214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63be00640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c1a3f10 0x7ff63c1a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 -- 172.21.15.27:0/2082706214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c10e2e0 msgr2=0x7ff63c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c10e2e0 0x7ff63c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 -- 172.21.15.27:0/2082706214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c1a3f10 msgr2=0x7ff63c1a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c1a3f10 0x7ff63c1a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 -- 172.21.15.27:0/2082706214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1a4a00 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff63b400640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c920 tx=0x7ff62c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c007c70 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff62c007e10 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1a4cf0 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.915+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c018710 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.916+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1ac4e0 con 0x7ff63c109210 2024-04-26T21:23:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.917+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c069c10 con 0x7ff63c109210 2024-04-26T21:23:16.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.919+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff62c00f450 con 0x7ff63c109210 2024-04-26T21:23:16.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.920+0000 7ff638c00640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 0x7ff6200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:16.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.921+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7ff62c0d0b50 con 0x7ff63c109210 2024-04-26T21:23:16.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.921+0000 7ff633e00640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 0x7ff6200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:16.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.922+0000 7ff633e00640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 0x7ff6200a61e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff624009230 tx=0x7ff624000bc0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:16.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:16.925+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff62c014030 con 0x7ff63c109210 2024-04-26T21:23:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.187+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63c111540 con 0x7ff63c109210 2024-04-26T21:23:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.188+0000 7ff638c00640 1 -- 172.21.15.27:0/2082706214 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11919 (secure 0 0 0) 0x7ff62c01f300 con 0x7ff63c109210 2024-04-26T21:23:17.189 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:23:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:12:13.355287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[14640,34406,34517,34472,34430]},"id":1}]} 2024-04-26T21:23:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 msgr2=0x7ff6200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 0x7ff6200a61e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff624009230 tx=0x7ff624000bc0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 msgr2=0x7ff63c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c920 tx=0x7ff62c00cde0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 shutdown_connections 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7ff6200a3d20 0x7ff6200a61e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c10e2e0 0x7ff63c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff63c1a3f10 0x7ff63c1a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 --2- 172.21.15.27:0/2082706214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff63c109210 0x7ff63c19f6a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 >> 172.21.15.27:0/2082706214 conn(0x7ff63c076f50 msgr2=0x7ff63c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 shutdown_connections 2024-04-26T21:23:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.191+0000 7ff642000640 1 -- 172.21.15.27:0/2082706214 wait complete. 2024-04-26T21:23:17.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.431+0000 7f5d42200640 1 -- 172.21.15.27:0/2838381171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c079440 msgr2=0x7f5d3c113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.431+0000 7f5d42200640 1 --2- 172.21.15.27:0/2838381171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c079440 0x7f5d3c113050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009900 tx=0x7f5d2802a410 comp rx=0 tx=0).stop 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 -- 172.21.15.27:0/2838381171 shutdown_connections 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 --2- 172.21.15.27:0/2838381171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c113590 0x7f5d3c115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 --2- 172.21.15.27:0/2838381171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c079440 0x7f5d3c113050 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 --2- 172.21.15.27:0/2838381171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c078a90 0x7f5d3c078e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 -- 172.21.15.27:0/2838381171 >> 172.21.15.27:0/2838381171 conn(0x7f5d3c1003a0 msgr2=0x7f5d3c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 -- 172.21.15.27:0/2838381171 shutdown_connections 2024-04-26T21:23:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.432+0000 7f5d42200640 1 -- 172.21.15.27:0/2838381171 wait complete. 2024-04-26T21:23:17.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 Processor -- start 2024-04-26T21:23:17.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 -- start start 2024-04-26T21:23:17.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c078a90 0x7f5d3c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c079440 0x7f5d3c19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c118f10 con 0x7f5d3c113590 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c118d90 con 0x7f5d3c078a90 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.433+0000 7f5d42200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c119090 con 0x7f5d3c079440 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49366/0 (socket says 172.21.15.27:49366) 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 -- 172.21.15.27:0/4110524569 learned_addr learned my addr 172.21.15.27:0/4110524569 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d40e00640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c078a90 0x7f5d3c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d3be00640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c079440 0x7f5d3c19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 -- 172.21.15.27:0/4110524569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c079440 msgr2=0x7f5d3c19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c079440 0x7f5d3c19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 -- 172.21.15.27:0/4110524569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c078a90 msgr2=0x7f5d3c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c078a90 0x7f5d3c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 -- 172.21.15.27:0/4110524569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a4540 con 0x7f5d3c113590 2024-04-26T21:23:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.434+0000 7f5d41800640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009fd0 tx=0x7f5d3400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:17.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.435+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017070 con 0x7f5d3c113590 2024-04-26T21:23:17.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.435+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d34003e00 con 0x7f5d3c113590 2024-04-26T21:23:17.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.435+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a4830 con 0x7f5d3c113590 2024-04-26T21:23:17.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.435+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34010460 con 0x7f5d3c113590 2024-04-26T21:23:17.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.435+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1ac0c0 con 0x7f5d3c113590 2024-04-26T21:23:17.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.436+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c069ac0 con 0x7f5d3c113590 2024-04-26T21:23:17.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.438+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5d34003970 con 0x7f5d3c113590 2024-04-26T21:23:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.440+0000 7f5d39600640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 0x7f5d240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.440+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f5d340d0960 con 0x7f5d3c113590 2024-04-26T21:23:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.440+0000 7f5d40e00640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 0x7f5d240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.441+0000 7f5d40e00640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 0x7f5d240a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d3000b390 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.444+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5d3409cf30 con 0x7f5d3c113590 2024-04-26T21:23:17.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:17 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2082706214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:17.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.692+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d3c10e190 con 0x7f5d3c113590 2024-04-26T21:23:17.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.693+0000 7f5d39600640 1 -- 172.21.15.27:0/4110524569 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11919 (secure 0 0 0) 0x7f5d3409c830 con 0x7f5d3c113590 2024-04-26T21:23:17.694 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:23:17.695 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2990870611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2990870611},{"type":"v1","addr":"172.21.15.27:6835","nonce":2990870611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1830505768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1830505768},{"type":"v1","addr":"172.21.15.27:6837","nonce":1830505768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:09:40.863524+0000","modified":"2024-04-26T21:12:13.355287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34430,"mds_3":34406,"mds_4":14640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/4118119230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":4118119230},{"type":"v1","addr":"172.21.15.27:6839","nonce":4118119230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6841/1078046722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1078046722},{"type":"v1","addr":"172.21.15.27:6841","nonce":1078046722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3519578339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3519578339},{"type":"v1","addr":"172.21.15.204:6833","nonce":3519578339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1241203213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1241203213},{"type":"v1","addr":"172.21.15.204:6835","nonce":1241203213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/743097112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":743097112},{"type":"v1","addr":"172.21.15.204:6837","nonce":743097112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6839/1122553339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1122553339},{"type":"v1","addr":"172.21.15.204:6839","nonce":1122553339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.94:6833/2182356287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2182356287},{"type":"v1","addr":"172.21.15.94:6833","nonce":2182356287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.94:6835/410689838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":410689838},{"type":"v1","addr":"172.21.15.94:6835","nonce":410689838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.94:6837/396354174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":396354174},{"type":"v1","addr":"172.21.15.94:6837","nonce":396354174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/12233135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":12233135},{"type":"v1","addr":"172.21.15.94:6839","nonce":12233135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[14640,34406,34517,34472,34430]},"id":1}]} 2024-04-26T21:23:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.695+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 msgr2=0x7f5d240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 0x7f5d240a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d3000b390 comp rx=0 tx=0).stop 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 msgr2=0x7f5d3c1a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009fd0 tx=0x7f5d3400b260 comp rx=0 tx=0).stop 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 shutdown_connections 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f5d240a3da0 0x7f5d240a6260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3c079440 0x7f5d3c19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d3c113590 0x7f5d3c1a3e10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 --2- 172.21.15.27:0/4110524569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c078a90 0x7f5d3c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 >> 172.21.15.27:0/4110524569 conn(0x7f5d3c1003a0 msgr2=0x7f5d3c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 shutdown_connections 2024-04-26T21:23:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.696+0000 7f5d42200640 1 -- 172.21.15.27:0/4110524569 wait complete. 2024-04-26T21:23:17.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:23:17.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:17 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2082706214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:17.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:17 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2082706214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:17.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 -- 172.21.15.27:0/3789127516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 msgr2=0x7fc8bc10ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/3789127516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc10ad10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009930 tx=0x7fc8ac02a420 comp rx=0 tx=0).stop 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 -- 172.21.15.27:0/3789127516 shutdown_connections 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/3789127516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 0x7fc8bc1117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/3789127516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 0x7fc8bc104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/3789127516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc10ad10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 -- 172.21.15.27:0/3789127516 >> 172.21.15.27:0/3789127516 conn(0x7fc8bc1003d0 msgr2=0x7fc8bc1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 -- 172.21.15.27:0/3789127516 shutdown_connections 2024-04-26T21:23:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.936+0000 7fc8c1a00640 1 -- 172.21.15.27:0/3789127516 wait complete. 2024-04-26T21:23:17.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 Processor -- start 2024-04-26T21:23:17.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 -- start start 2024-04-26T21:23:17.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 0x7fc8bc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 0x7fc8bc19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114ac0 con 0x7fc8bc10a930 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114940 con 0x7fc8bc104920 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.938+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114c40 con 0x7fc8bc1052e0 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49376/0 (socket says 172.21.15.27:49376) 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 -- 172.21.15.27:0/2692058401 learned_addr learned my addr 172.21.15.27:0/2692058401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8bbe00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 0x7fc8bc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8b3e00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 0x7fc8bc19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 -- 172.21.15.27:0/2692058401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 msgr2=0x7fc8bc19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 0x7fc8bc19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 -- 172.21.15.27:0/2692058401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 msgr2=0x7fc8bc19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:17.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 0x7fc8bc19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:17.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.939+0000 7fc8c1000640 1 -- 172.21.15.27:0/2692058401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc1a0070 con 0x7fc8bc10a930 2024-04-26T21:23:17.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8c1000640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400eaa0 tx=0x7fc8b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b400cdb0 con 0x7fc8bc10a930 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b4005020 con 0x7fc8bc10a930 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40116d0 con 0x7fc8bc10a930 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc1a0360 con 0x7fc8bc10a930 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.940+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1a7bf0 con 0x7fc8bc10a930 2024-04-26T21:23:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.941+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc069ac0 con 0x7fc8bc10a930 2024-04-26T21:23:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.944+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc8b40051d0 con 0x7fc8bc10a930 2024-04-26T21:23:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.945+0000 7fc8b9600640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 0x7fc8a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.945+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7fc8b40d0a00 con 0x7fc8bc10a930 2024-04-26T21:23:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.946+0000 7fc8bbe00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 0x7fc8a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.946+0000 7fc8bbe00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 0x7fc8a00a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0057c0 tx=0x7fc8ac005a40 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:17.949+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc8b4016030 con 0x7fc8bc10a930 2024-04-26T21:23:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.158+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8bc02dad0 con 0x7fc8bc10a930 2024-04-26T21:23:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.161+0000 7fc8b9600640 1 -- 172.21.15.27:0/2692058401 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23267 (secure 0 0 0) 0x7fc8b409d030 con 0x7fc8bc10a930 2024-04-26T21:23:18.163 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-26T21:23:18.163 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"a125c556-040f-11ef-bc93-c7b262605968","created":"2024-04-26T20:58:51.280981+0000","modified":"2024-04-26T21:23:11.351982+0000","last_up_change":"2024-04-26T21:08:36.456131+0000","last_in_change":"2024-04-26T21:08:10.907810+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:03:22.088553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:38.486492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:39.830620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa98cfe0-26a3-4f94-bc02-6c01411e6ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6803","nonce":530549701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6805","nonce":530549701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6809","nonce":530549701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":530549701},{"type":"v1","addr":"172.21.15.27:6807","nonce":530549701}]},"public_addr":"172.21.15.27:6803/530549701","cluster_addr":"172.21.15.27:6805/530549701","heartbeat_back_addr":"172.21.15.27:6809/530549701","heartbeat_front_addr":"172.21.15.27:6807/530549701","state":["exists","up"]},{"osd":1,"uuid":"b88f3050-7a38-4f9e-8719-47deef44eb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6801","nonce":3294273778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6803","nonce":3294273778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6807","nonce":3294273778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3294273778},{"type":"v1","addr":"172.21.15.94:6805","nonce":3294273778}]},"public_addr":"172.21.15.94:6801/3294273778","cluster_addr":"172.21.15.94:6803/3294273778","heartbeat_back_addr":"172.21.15.94:6807/3294273778","heartbeat_front_addr":"172.21.15.94:6805/3294273778","state":["exists","up"]},{"osd":2,"uuid":"689b4746-40ea-4411-bdb5-2aeedd58b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6801","nonce":3559310933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6803","nonce":3559310933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6807","nonce":3559310933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3559310933},{"type":"v1","addr":"172.21.15.204:6805","nonce":3559310933}]},"public_addr":"172.21.15.204:6801/3559310933","cluster_addr":"172.21.15.204:6803/3559310933","heartbeat_back_addr":"172.21.15.204:6807/3559310933","heartbeat_front_addr":"172.21.15.204:6805/3559310933","state":["exists","up"]},{"osd":3,"uuid":"496145b1-6477-4d0e-a222-c6222e95eac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6811","nonce":2585963482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6813","nonce":2585963482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6817","nonce":2585963482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2585963482},{"type":"v1","addr":"172.21.15.27:6815","nonce":2585963482}]},"public_addr":"172.21.15.27:6811/2585963482","cluster_addr":"172.21.15.27:6813/2585963482","heartbeat_back_addr":"172.21.15.27:6817/2585963482","heartbeat_front_addr":"172.21.15.27:6815/2585963482","state":["exists","up"]},{"osd":4,"uuid":"c62db123-2dc9-4592-ad16-f2fe0723e9cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6809","nonce":128441754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6811","nonce":128441754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6815","nonce":128441754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":128441754},{"type":"v1","addr":"172.21.15.94:6813","nonce":128441754}]},"public_addr":"172.21.15.94:6809/128441754","cluster_addr":"172.21.15.94:6811/128441754","heartbeat_back_addr":"172.21.15.94:6815/128441754","heartbeat_front_addr":"172.21.15.94:6813/128441754","state":["exists","up"]},{"osd":5,"uuid":"380e7876-c897-462a-96ee-0f6ef1239fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6809","nonce":4134827726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6811","nonce":4134827726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6815","nonce":4134827726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4134827726},{"type":"v1","addr":"172.21.15.204:6813","nonce":4134827726}]},"public_addr":"172.21.15.204:6809/4134827726","cluster_addr":"172.21.15.204:6811/4134827726","heartbeat_back_addr":"172.21.15.204:6815/4134827726","heartbeat_front_addr":"172.21.15.204:6813/4134827726","state":["exists","up"]},{"osd":6,"uuid":"a2c9acfd-f6a3-41e6-a3d3-b694781bd208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6819","nonce":1091396972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6821","nonce":1091396972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6825","nonce":1091396972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1091396972},{"type":"v1","addr":"172.21.15.27:6823","nonce":1091396972}]},"public_addr":"172.21.15.27:6819/1091396972","cluster_addr":"172.21.15.27:6821/1091396972","heartbeat_back_addr":"172.21.15.27:6825/1091396972","heartbeat_front_addr":"172.21.15.27:6823/1091396972","state":["exists","up"]},{"osd":7,"uuid":"f1776a60-ed45-4ece-8668-fe147a04409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6817","nonce":1853319664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6819","nonce":1853319664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6823","nonce":1853319664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1853319664},{"type":"v1","addr":"172.21.15.94:6821","nonce":1853319664}]},"public_addr":"172.21.15.94:6817/1853319664","cluster_addr":"172.21.15.94:6819/1853319664","heartbeat_back_addr":"172.21.15.94:6823/1853319664","heartbeat_front_addr":"172.21.15.94:6821/1853319664","state":["exists","up"]},{"osd":8,"uuid":"3c87f761-8ce7-4634-92e9-223f67e64af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6817","nonce":3522528239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6819","nonce":3522528239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6823","nonce":3522528239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3522528239},{"type":"v1","addr":"172.21.15.204:6821","nonce":3522528239}]},"public_addr":"172.21.15.204:6817/3522528239","cluster_addr":"172.21.15.204:6819/3522528239","heartbeat_back_addr":"172.21.15.204:6823/3522528239","heartbeat_front_addr":"172.21.15.204:6821/3522528239","state":["exists","up"]},{"osd":9,"uuid":"53a783f5-a861-4ea2-b77f-f54e79b19da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6827","nonce":2596688206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6829","nonce":2596688206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6833","nonce":2596688206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2596688206},{"type":"v1","addr":"172.21.15.27:6831","nonce":2596688206}]},"public_addr":"172.21.15.27:6827/2596688206","cluster_addr":"172.21.15.27:6829/2596688206","heartbeat_back_addr":"172.21.15.27:6833/2596688206","heartbeat_front_addr":"172.21.15.27:6831/2596688206","state":["exists","up"]},{"osd":10,"uuid":"70efbe0c-9374-43d6-b479-f1412debf3e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6825","nonce":141255194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6827","nonce":141255194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6831","nonce":141255194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":141255194},{"type":"v1","addr":"172.21.15.94:6829","nonce":141255194}]},"public_addr":"172.21.15.94:6825/141255194","cluster_addr":"172.21.15.94:6827/141255194","heartbeat_back_addr":"172.21.15.94:6831/141255194","heartbeat_front_addr":"172.21.15.94:6829/141255194","state":["exists","up"]},{"osd":11,"uuid":"c543ea6d-1e49-42e8-aea5-1cea36e42415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6825","nonce":1542928815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6827","nonce":1542928815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6831","nonce":1542928815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1542928815},{"type":"v1","addr":"172.21.15.204:6829","nonce":1542928815}]},"public_addr":"172.21.15.204:6825/1542928815","cluster_addr":"172.21.15.204:6827/1542928815","heartbeat_back_addr":"172.21.15.204:6831/1542928815","heartbeat_front_addr":"172.21.15.204:6829/1542928815","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:13.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:02:47.145992+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:20.157740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:03:55.840777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:04:29.327424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:02.990137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:05:39.041579+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:13.336852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:06:47.849790+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:24.996314+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:07:59.714556+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:33.711091+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2522153056":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/3447033143":"2024-04-27T21:00:01.870639+0000","172.21.15.27:6800/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6800/128242469":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2239698495":"2024-04-27T20:59:25.698004+0000","172.21.15.27:6801/2094082344":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/1042211156":"2024-04-27T20:59:25.698004+0000","172.21.15.27:0/2901658935":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/2516445598":"2024-04-27T21:00:01.870639+0000","172.21.15.27:0/542903082":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6800/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:6801/3365757774":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/2356002952":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/137865064":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/1196187505":"2024-04-27T21:12:12.895075+0000","172.21.15.27:0/238118775":"2024-04-27T21:12:12.895075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:23:18.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 msgr2=0x7fc8a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 0x7fc8a00a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0057c0 tx=0x7fc8ac005a40 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 msgr2=0x7fc8bc19f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400eaa0 tx=0x7fc8b400ef60 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 shutdown_connections 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fc8a00a3d20 0x7fc8a00a61e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8bc1052e0 0x7fc8bc19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc10a930 0x7fc8bc19f940 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 --2- 172.21.15.27:0/2692058401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8bc104920 0x7fc8bc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 >> 172.21.15.27:0/2692058401 conn(0x7fc8bc1003d0 msgr2=0x7fc8bc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 shutdown_connections 2024-04-26T21:23:18.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.164+0000 7fc8c1a00640 1 -- 172.21.15.27:0/2692058401 wait complete. 2024-04-26T21:23:18.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 -- 172.21.15.27:0/2327259593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 msgr2=0x7f570c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 --2- 172.21.15.27:0/2327259593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c104d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009930 tx=0x7f56f802a420 comp rx=0 tx=0).stop 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 -- 172.21.15.27:0/2327259593 shutdown_connections 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 --2- 172.21.15.27:0/2327259593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 0x7f570c1117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 --2- 172.21.15.27:0/2327259593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c104d90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 --2- 172.21.15.27:0/2327259593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 0x7f570c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 -- 172.21.15.27:0/2327259593 >> 172.21.15.27:0/2327259593 conn(0x7f570c1003a0 msgr2=0x7f570c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 -- 172.21.15.27:0/2327259593 shutdown_connections 2024-04-26T21:23:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.406+0000 7f570be00640 1 -- 172.21.15.27:0/2327259593 wait complete. 2024-04-26T21:23:18.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.407+0000 7f570be00640 1 Processor -- start 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.407+0000 7f570be00640 1 -- start start 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 0x7f570c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 0x7f570c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f570c114a60 con 0x7f570c104910 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f570c1148e0 con 0x7f570c1052d0 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f570c114be0 con 0x7f570c10a920 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49396/0 (socket says 172.21.15.27:49396) 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570aa00640 1 -- 172.21.15.27:0/2863521127 learned_addr learned my addr 172.21.15.27:0/2863521127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570a000640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 0x7f570c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570b400640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 0x7f570c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570aa00640 1 -- 172.21.15.27:0/2863521127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 msgr2=0x7f570c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.408+0000 7f570aa00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 0x7f570c19f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570aa00640 1 -- 172.21.15.27:0/2863521127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 msgr2=0x7f570c19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570aa00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 0x7f570c19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570aa00640 1 -- 172.21.15.27:0/2863521127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c1a0070 con 0x7f570c104910 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570aa00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d950 tx=0x7f56fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc017070 con 0x7f570c104910 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56fc009c30 con 0x7f570c104910 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00f620 con 0x7f570c104910 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c1a0360 con 0x7f570c104910 2024-04-26T21:23:18.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.409+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c1a7bf0 con 0x7f570c104910 2024-04-26T21:23:18.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.410+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570c069ac0 con 0x7f570c104910 2024-04-26T21:23:18.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.412+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f56fc010460 con 0x7f570c104910 2024-04-26T21:23:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.414+0000 7f5703400640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 0x7f56f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.414+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f56fc0d04b0 con 0x7f570c104910 2024-04-26T21:23:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.414+0000 7f570a000640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 0x7f56f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.415+0000 7f570a000640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 0x7f56f00a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006010 tx=0x7f56f8009610 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.418+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f56fc09ca80 con 0x7f570c104910 2024-04-26T21:23:18.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-mon[28085]: pgmap v344: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:18.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/4110524569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.485 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2692058401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.671+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f570c02dad0 con 0x7f570c104910 2024-04-26T21:23:18.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.677+0000 7f5703400640 1 -- 172.21.15.27:0/2863521127 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f56fc009de0 con 0x7f570c104910 2024-04-26T21:23:18.679 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:23:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 msgr2=0x7f56f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 0x7f56f00a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006010 tx=0x7f56f8009610 comp rx=0 tx=0).stop 2024-04-26T21:23:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 msgr2=0x7f570c19b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d950 tx=0x7f56fc00de10 comp rx=0 tx=0).stop 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 shutdown_connections 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f56f00a3da0 0x7f56f00a6260 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006010 tx=0x7f56f8009610 comp rx=0 tx=0).stop 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570c10a920 0x7f570c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f570c1052d0 0x7f570c19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 --2- 172.21.15.27:0/2863521127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f570c104910 0x7f570c19b080 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.682+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 >> 172.21.15.27:0/2863521127 conn(0x7f570c1003a0 msgr2=0x7f570c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.683+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 shutdown_connections 2024-04-26T21:23:18.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.683+0000 7f570be00640 1 -- 172.21.15.27:0/2863521127 wait complete. 2024-04-26T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:23:18.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mon[32145]: pgmap v344: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/4110524569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2692058401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mon[31273]: pgmap v344: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:23:18.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/4110524569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2692058401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 -- 172.21.15.27:0/4126556335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c104b30 msgr2=0x7fdf3c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 --2- 172.21.15.27:0/4126556335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c104b30 0x7fdf3c104fb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009900 tx=0x7fdf2802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 -- 172.21.15.27:0/4126556335 shutdown_connections 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 --2- 172.21.15.27:0/4126556335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c1119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 --2- 172.21.15.27:0/4126556335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c104b30 0x7fdf3c104fb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 --2- 172.21.15.27:0/4126556335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 0x7fdf3c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 -- 172.21.15.27:0/4126556335 >> 172.21.15.27:0/4126556335 conn(0x7fdf3c1005a0 msgr2=0x7fdf3c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 -- 172.21.15.27:0/4126556335 shutdown_connections 2024-04-26T21:23:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.915+0000 7fdf44400640 1 -- 172.21.15.27:0/4126556335 wait complete. 2024-04-26T21:23:18.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.916+0000 7fdf44400640 1 Processor -- start 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.916+0000 7fdf44400640 1 -- start start 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 0x7fdf3c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 0x7fdf3c19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c114c90 con 0x7fdf3c1054f0 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c114b10 con 0x7fdf3c10abf0 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf44400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c114e10 con 0x7fdf3c104b30 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf41000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf41000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49448/0 (socket says 172.21.15.27:49448) 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf41000640 1 -- 172.21.15.27:0/1662996765 learned_addr learned my addr 172.21.15.27:0/1662996765 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf41a00640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 0x7fdf3c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.917+0000 7fdf42400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 0x7fdf3c19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 -- 172.21.15.27:0/1662996765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 msgr2=0x7fdf3c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 0x7fdf3c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 -- 172.21.15.27:0/1662996765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 msgr2=0x7fdf3c19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 0x7fdf3c19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 -- 172.21.15.27:0/1662996765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c1a01c0 con 0x7fdf3c1054f0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41a00640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 0x7fdf3c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf41000640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf28006010 tx=0x7fdf280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2803f030 con 0x7fdf3c1054f0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf28003c90 con 0x7fdf3c1054f0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c1a0450 con 0x7fdf3c1054f0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28035070 con 0x7fdf3c1054f0 2024-04-26T21:23:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.918+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c1a7d40 con 0x7fdf3c1054f0 2024-04-26T21:23:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.920+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c069750 con 0x7fdf3c1054f0 2024-04-26T21:23:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.922+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdf28042030 con 0x7fdf3c1054f0 2024-04-26T21:23:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.923+0000 7fdf3a000640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 0x7fdf240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.923+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9023+0+0 (secure 0 0 0) 0x7fdf280f0fd0 con 0x7fdf3c1054f0 2024-04-26T21:23:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.923+0000 7fdf41a00640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 0x7fdf240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:18.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.924+0000 7fdf41a00640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 0x7fdf240a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf300058c0 tx=0x7fdf30005850 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:18.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:18.928+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf280bd420 con 0x7fdf3c1054f0 2024-04-26T21:23:18.985 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:18.986 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:18.986 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-mds[69648]: -- 172.21.15.27:0/261083368 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55678e5e5a40 con 0x55678e5b9000 2024-04-26T21:23:18.986 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: starting mds.g at 2024-04-26T21:23:18.987 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a[28081]: 2024-04-26T21:23:18.673+0000 7fdbac400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:23:18.987 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:18.988 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:18.988 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-mds[70032]: -- 172.21.15.27:0/4254747839 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55634cee3a40 con 0x55634ceb6800 2024-04-26T21:23:18.988 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:23:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: starting mds.a at 2024-04-26T21:23:19.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.186+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdf3c10cf70 con 0x7fdf3c1054f0 2024-04-26T21:23:19.253 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.253 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.253 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mds[62744]: -- 172.21.15.204:0/566704593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607e3681a40 con 0x5607e3654c00 2024-04-26T21:23:19.253 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: starting mds.c at 2024-04-26T21:23:19.253 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mds[62431]: -- 172.21.15.204:0/2879413052 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ace6b9da40 con 0x55ace6b70c00 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: starting mds.f at 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.254 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mds[63047]: -- 172.21.15.204:0/1213786029 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56222e869a40 con 0x56222e83c800 2024-04-26T21:23:19.255 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: starting mds.i at 2024-04-26T21:23:19.255 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.255 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.255 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-mds[62127]: -- 172.21.15.204:0/12245553 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56435d8f5a40 con 0x56435d8c8c00 2024-04-26T21:23:19.255 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:23:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: starting mds.l at 2024-04-26T21:23:19.276 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.276 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mds[62177]: -- 172.21.15.94:0/3066492448 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd9f6dda40 con 0x55fd9f6b1000 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: starting mds.b at 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mds[61557]: -- 172.21.15.94:0/635801601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab7e301a40 con 0x55ab7e2d4800 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: starting mds.e at 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.277 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mds[61255]: -- 172.21.15.94:0/4144300558 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56480e80fa40 con 0x56480e7e3000 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: starting mds.k at 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: ignoring --setuser ceph since I am not root 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: ignoring --setgroup ceph since I am not root 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-mds[61865]: -- 172.21.15.94:0/1668259909 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600500cfa40 con 0x5600500a2800 2024-04-26T21:23:19.278 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:23:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: starting mds.h at 2024-04-26T21:23:19.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.681+0000 7fdf3a000640 1 -- 172.21.15.27:0/1662996765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fdf280bcd20 con 0x7fdf3c1054f0 2024-04-26T21:23:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 msgr2=0x7fdf240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 0x7fdf240a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf300058c0 tx=0x7fdf30005850 comp rx=0 tx=0).stop 2024-04-26T21:23:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 msgr2=0x7fdf3c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf28006010 tx=0x7fdf280042f0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 shutdown_connections 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fdf240a3d20 0x7fdf240a61e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf3c10abf0 0x7fdf3c19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf3c1054f0 0x7fdf3c19b710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 --2- 172.21.15.27:0/1662996765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf3c104b30 0x7fdf3c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 >> 172.21.15.27:0/1662996765 conn(0x7fdf3c1005a0 msgr2=0x7fdf3c1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 shutdown_connections 2024-04-26T21:23:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.686+0000 7fdf44400640 1 -- 172.21.15.27:0/1662996765 wait complete. 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T21:23:19.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:19 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:23:19.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:23:19.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:23:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:23:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:23:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:23:19.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T21:23:19.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:23:19.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T21:23:19.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:19 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:23:19.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/2863521127' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T21:23:19.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:19 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 -- 172.21.15.27:0/2920717781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 msgr2=0x7f9d08077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 --2- 172.21.15.27:0/2920717781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d08077240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009900 tx=0x7f9cf802a410 comp rx=0 tx=0).stop 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 -- 172.21.15.27:0/2920717781 shutdown_connections 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 --2- 172.21.15.27:0/2920717781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08113640 0x7f9d08115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 --2- 172.21.15.27:0/2920717781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08077780 0x7f9d08113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 --2- 172.21.15.27:0/2920717781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d08077240 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 -- 172.21.15.27:0/2920717781 >> 172.21.15.27:0/2920717781 conn(0x7f9d08100490 msgr2=0x7f9d081028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 -- 172.21.15.27:0/2920717781 shutdown_connections 2024-04-26T21:23:19.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.929+0000 7f9d07200640 1 -- 172.21.15.27:0/2920717781 wait complete. 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 Processor -- start 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 -- start start 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08077780 0x7f9d0819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 0x7f9d081a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08118f60 con 0x7f9d08076e60 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08118de0 con 0x7f9d08113640 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.931+0000 7f9d07200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081190e0 con 0x7f9d08077780 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39504/0 (socket says 172.21.15.27:39504) 2024-04-26T21:23:19.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 -- 172.21.15.27:0/853011161 learned_addr learned my addr 172.21.15.27:0/853011161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05400640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08077780 0x7f9d0819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d06800640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 0x7f9d081a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 -- 172.21.15.27:0/853011161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08077780 msgr2=0x7f9d0819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08077780 0x7f9d0819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 -- 172.21.15.27:0/853011161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 msgr2=0x7f9d081a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 0x7f9d081a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d05e00640 1 -- 172.21.15.27:0/853011161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d081a44e0 con 0x7f9d08076e60 2024-04-26T21:23:19.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.932+0000 7f9d06800640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 0x7f9d081a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9d05e00640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8004770 tx=0x7f9cf8003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf803f070 con 0x7f9d08076e60 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cf803bd00 con 0x7f9d08076e60 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d081a4770 con 0x7f9d08076e60 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8035030 con 0x7f9d08076e60 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.933+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d081ac060 con 0x7f9d08076e60 2024-04-26T21:23:19.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.934+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d08069ac0 con 0x7f9d08076e60 2024-04-26T21:23:19.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.936+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9cf8042070 con 0x7f9d08076e60 2024-04-26T21:23:19.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.939+0000 7f9cfea00640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 0x7f9cec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:19.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.939+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9023+0+0 (secure 0 0 0) 0x7f9cf80f1280 con 0x7f9d08076e60 2024-04-26T21:23:19.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.939+0000 7f9d05400640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 0x7f9cec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:19.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.940+0000 7f9d05400640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 0x7f9cec0a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d08102960 tx=0x7f9cf400b590 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:19.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:19.944+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9cf80bd6d0 con 0x7f9d08076e60 2024-04-26T21:23:20.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.158+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9d0810e7d0 con 0x7f9d08076e60 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: pgmap v347: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: Cluster is now healthy 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6838/184137892,v1:172.21.15.27:6839/184137892] up:boot 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.27:6840/885056458,v1:172.21.15.27:6841/885056458] up:boot 2024-04-26T21:23:20.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6832/2862575824,v1:172.21.15.204:6834/2862575824] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6833/4165904465,v1:172.21.15.204:6835/4165904465] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6832/1815904326,v1:172.21.15.94:6834/1815904326] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6833/2141574743,v1:172.21.15.94:6835/2141574743] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6836/62778421,v1:172.21.15.94:6837/62778421] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.94:6838/3818625039,v1:172.21.15.94:6839/3818625039] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6836/2619014055,v1:172.21.15.204:6838/2619014055] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: mds.? [v2:172.21.15.204:6837/3780217578,v1:172.21.15.204:6839/3780217578] up:boot 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: fsmap 12 up:standby 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:23:20.738 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:23:20.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:20 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:20.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: pgmap v347: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-26T21:23:20.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:23:20.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:23:20.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: Cluster is now healthy 2024-04-26T21:23:20.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6838/184137892,v1:172.21.15.27:6839/184137892] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.27:6840/885056458,v1:172.21.15.27:6841/885056458] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6832/2862575824,v1:172.21.15.204:6834/2862575824] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6833/4165904465,v1:172.21.15.204:6835/4165904465] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6832/1815904326,v1:172.21.15.94:6834/1815904326] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6833/2141574743,v1:172.21.15.94:6835/2141574743] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6836/62778421,v1:172.21.15.94:6837/62778421] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.94:6838/3818625039,v1:172.21.15.94:6839/3818625039] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6836/2619014055,v1:172.21.15.204:6838/2619014055] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: mds.? [v2:172.21.15.204:6837/3780217578,v1:172.21.15.204:6839/3780217578] up:boot 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: fsmap 12 up:standby 2024-04-26T21:23:20.754 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:23:20.755 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:20 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: pgmap v347: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: Cluster is now healthy 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/1662996765' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6838/184137892,v1:172.21.15.27:6839/184137892] up:boot 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.27:6840/885056458,v1:172.21.15.27:6841/885056458] up:boot 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6832/2862575824,v1:172.21.15.204:6834/2862575824] up:boot 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6833/4165904465,v1:172.21.15.204:6835/4165904465] up:boot 2024-04-26T21:23:20.777 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6832/1815904326,v1:172.21.15.94:6834/1815904326] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6833/2141574743,v1:172.21.15.94:6835/2141574743] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6836/62778421,v1:172.21.15.94:6837/62778421] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.94:6838/3818625039,v1:172.21.15.94:6839/3818625039] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6836/2619014055,v1:172.21.15.204:6838/2619014055] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: mds.? [v2:172.21.15.204:6837/3780217578,v1:172.21.15.204:6839/3780217578] up:boot 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: fsmap 12 up:standby 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:23:20.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:23:20.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:23:20.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:23:20.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='mgr.15342 172.21.15.27:0/2164287745' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:23:20.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:20 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:20.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.801+0000 7f9cfea00640 1 -- 172.21.15.27:0/853011161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f9cf80bcfd0 con 0x7f9d08076e60 2024-04-26T21:23:20.803 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:23:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.806+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 msgr2=0x7f9cec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.806+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 0x7f9cec0a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d08102960 tx=0x7f9cf400b590 comp rx=0 tx=0).stop 2024-04-26T21:23:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.806+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 msgr2=0x7f9d0819f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.806+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8004770 tx=0x7f9cf8003ca0 comp rx=0 tx=0).stop 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 shutdown_connections 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7f9cec0a3d20 0x7f9cec0a61e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d08077780 0x7f9d0819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d08113640 0x7f9d081a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 --2- 172.21.15.27:0/853011161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d08076e60 0x7f9d0819f540 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.807+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 >> 172.21.15.27:0/853011161 conn(0x7f9d08100490 msgr2=0x7f9d08114d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.809+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 shutdown_connections 2024-04-26T21:23:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:20.809+0000 7f9d07200640 1 -- 172.21.15.27:0/853011161 wait complete. 2024-04-26T21:23:20.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:23:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 -- 172.21.15.27:0/426558575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac1042b0 msgr2=0x7fe5ac10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/426558575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac10c790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe594009900 tx=0x7fe59402a480 comp rx=0 tx=0).stop 2024-04-26T21:23:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 -- 172.21.15.27:0/426558575 shutdown_connections 2024-04-26T21:23:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/426558575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/426558575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac10c790 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/426558575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac103990 0x7fe5ac103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 -- 172.21.15.27:0/426558575 >> 172.21.15.27:0/426558575 conn(0x7fe5ac0fd1b0 msgr2=0x7fe5ac0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 -- 172.21.15.27:0/426558575 shutdown_connections 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.051+0000 7fe5b1a00640 1 -- 172.21.15.27:0/426558575 wait complete. 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.052+0000 7fe5b1a00640 1 Processor -- start 2024-04-26T21:23:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.052+0000 7fe5b1a00640 1 -- start start 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac103990 0x7fe5ac19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac1129a0 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac112820 con 0x7fe5ac1042b0 2024-04-26T21:23:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac112b20 con 0x7fe5ac103990 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39518/0 (socket says 172.21.15.27:39518) 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 -- 172.21.15.27:0/3741612139 learned_addr learned my addr 172.21.15.27:0/3741612139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5abe00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac103990 0x7fe5ac19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5a3400640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 -- 172.21.15.27:0/3741612139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac103990 msgr2=0x7fe5ac19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac103990 0x7fe5ac19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 -- 172.21.15.27:0/3741612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac1042b0 msgr2=0x7fe5ac19b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.053+0000 7fe5b1000640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac19b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5b1000640 1 -- 172.21.15.27:0/3741612139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac1a00a0 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5b1000640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4011830 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a4016070 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac1a0390 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010d20 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.054+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac1a7c20 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.055+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac069670 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.058+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe5a401b460 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.059+0000 7fe5a9600640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 0x7fe5900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.059+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8417+0+0 (secure 0 0 0) 0x7fe5a40cfd10 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.060+0000 7fe5abe00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 0x7fe5900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.061+0000 7fe5abe00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 0x7fe5900a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007350 tx=0x7fe59c0072e0 comp rx=0 tx=0).ready entity=mgr.15342 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.064+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe5a40d5020 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.271+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5ac1a7eb0 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.806+0000 7fe5a9600640 1 -- 172.21.15.27:0/3741612139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7fe5a409c4c0 con 0x7fe5ac10cdb0 2024-04-26T21:23:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-04-26T21:23:21.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.810+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 msgr2=0x7fe5900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:21.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.810+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 0x7fe5900a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007350 tx=0x7fe59c0072e0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.810+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 msgr2=0x7fe5ac19f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:21.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.810+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a400b8e0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 shutdown_connections 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:6800/1450148262,v1:172.21.15.27:6801/1450148262] conn(0x7fe5900a3da0 0x7fe5900a6260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ac10cdb0 0x7fe5ac19f970 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5ac1042b0 0x7fe5ac19b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 --2- 172.21.15.27:0/3741612139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5ac103990 0x7fe5ac19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.811+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 >> 172.21.15.27:0/3741612139 conn(0x7fe5ac0fd1b0 msgr2=0x7fe5ac1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.812+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 shutdown_connections 2024-04-26T21:23:21.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-26T21:23:21.812+0000 7fe5b1a00640 1 -- 172.21.15.27:0/3741612139 wait complete. 2024-04-26T21:23:21.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:23:21.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:23:22.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:23:22.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:21 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:22.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:21 smithi204 ceph-mon[32145]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T21:23:22.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:21 smithi204 ceph-mon[32145]: pgmap v349: 65 pgs: 65 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:23:22.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:21 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:22.116 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T21:23:22.117 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:23:22.168 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:23:22.229 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:23:22.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:21 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:22.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:21 smithi027 ceph-mon[28085]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T21:23:22.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:21 smithi027 ceph-mon[28085]: pgmap v349: 65 pgs: 65 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:23:22.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:21 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:22.278 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:21 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/853011161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:22.278 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:21 smithi094 ceph-mon[31273]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T21:23:22.278 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:21 smithi094 ceph-mon[31273]: pgmap v349: 65 pgs: 65 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:23:22.278 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:21 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:23:22.279 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T21:23:22.279 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T21:23:22.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:23:22.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 systemd[1]: Stopping Ceph mon.a for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:23.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 ceph-mon[28085]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:23.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 ceph-mon[28085]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T21:23:23.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a[28081]: 2024-04-26T21:23:22.919+0000 7fdbb3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:23.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a[28081]: 2024-04-26T21:23:22.919+0000 7fdbb3200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T21:23:23.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:22 smithi027 podman[86625]: 2024-04-26 21:23:22.982159109 +0000 UTC m=+0.211406702 container died 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:23:23.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:22 smithi204 ceph-mon[32145]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:23.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:22 smithi204 ceph-mon[32145]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T21:23:23.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:22 smithi094 ceph-mon[31273]: from='client.? 172.21.15.27:0/3741612139' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:23:23.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:22 smithi094 ceph-mon[31273]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T21:23:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:23 smithi027 podman[86625]: 2024-04-26 21:23:23.470107644 +0000 UTC m=+0.699355273 container cleanup 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:23:23.735 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:23 smithi027 bash[86625]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:23:24.235 INFO:journalctl@ceph.mon.a.smithi027.stdout:Apr 26 21:23:23 smithi027 podman[86636]: 2024-04-26 21:23:23.812900837 +0000 UTC m=+0.821002259 container remove 0e6c7ddb8f6ee6ec5d0350e70c845335b53e9f6e08584c0c0b95e70141687bb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:23:24.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T21:23:24.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:24.493 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T21:23:24.493 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T21:23:24.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.b 2024-04-26T21:23:25.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:24 smithi094 systemd[1]: Stopping Ceph mon.b for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:25.489 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b[31269]: 2024-04-26T21:23:25.070+0000 7f2d21e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:25.489 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b[31269]: 2024-04-26T21:23:25.070+0000 7f2d21e00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:23:25.489 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 podman[67691]: 2024-04-26 21:23:25.214721391 +0000 UTC m=+0.323570093 container died db99f4544d221f72c8e896d492bdec68ce339d996226c16552fb01ef4eb09384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T21:23:25.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 podman[67691]: 2024-04-26 21:23:25.625315503 +0000 UTC m=+0.734164158 container cleanup db99f4544d221f72c8e896d492bdec68ce339d996226c16552fb01ef4eb09384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:23:25.776 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 bash[67691]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b 2024-04-26T21:23:26.223 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 26 21:23:25 smithi094 podman[67705]: 2024-04-26 21:23:25.868861056 +0000 UTC m=+0.643075555 container remove db99f4544d221f72c8e896d492bdec68ce339d996226c16552fb01ef4eb09384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T21:23:26.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T21:23:26.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:26.398 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T21:23:26.398 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T21:23:26.399 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.c 2024-04-26T21:23:26.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:26 smithi204 systemd[1]: Stopping Ceph mon.c for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c[32140]: 2024-04-26T21:23:26.945+0000 7ff1af600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c[32140]: 2024-04-26T21:23:26.945+0000 7ff1af600700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:23:27.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:27 smithi204 podman[68643]: 2024-04-26 21:23:27.10077278 +0000 UTC m=+0.298278439 container died 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T21:23:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:27 smithi204 podman[68643]: 2024-04-26 21:23:27.497010975 +0000 UTC m=+0.694516627 container cleanup 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T21:23:27.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:27 smithi204 bash[68643]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c 2024-04-26T21:23:28.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Apr 26 21:23:27 smithi204 podman[68657]: 2024-04-26 21:23:27.848953244 +0000 UTC m=+0.729998181 container remove 170f35147ffec0bdc1cfc3a422d6e6c1d5066332a88e35c666c8aaa433826f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T21:23:28.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T21:23:28.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:28.303 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T21:23:28.303 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T21:23:28.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:23:28.735 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:28 smithi027 systemd[1]: Stopping Ceph mgr.x for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:29.015 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:23:28.923+0000 7ff1af200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:29.016 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x[28308]: 2024-04-26T21:23:28.923+0000 7ff1af200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T21:23:29.376 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:29 smithi027 podman[86743]: 2024-04-26 21:23:29.016707406 +0000 UTC m=+0.212254591 container died eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T21:23:29.690 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:29 smithi027 podman[86743]: 2024-04-26 21:23:29.583526146 +0000 UTC m=+0.779073368 container cleanup eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:23:29.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:29 smithi027 bash[86743]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x 2024-04-26T21:23:29.984 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Apr 26 21:23:29 smithi027 podman[86757]: 2024-04-26 21:23:29.743221592 +0000 UTC m=+0.711657192 container remove eb89e39ef638ddc21b44e7447a072aad45e0566cc54fa4d155eb6cb11640f65d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T21:23:30.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T21:23:30.267 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:30.267 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T21:23:30.267 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T21:23:30.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.y 2024-04-26T21:23:30.776 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:30 smithi094 systemd[1]: Stopping Ceph mgr.y for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:31.206 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:30 smithi094 podman[67823]: 2024-04-26 21:23:30.906632363 +0000 UTC m=+0.233022886 container died c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T21:23:31.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:31 smithi094 podman[67823]: 2024-04-26 21:23:31.363691913 +0000 UTC m=+0.690082455 container cleanup c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T21:23:31.526 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:31 smithi094 bash[67823]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y 2024-04-26T21:23:32.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:31 smithi094 podman[67837]: 2024-04-26 21:23:31.648162524 +0000 UTC m=+0.727185930 container remove c7dbc05a7ad53b0afd9abfd2c0934f8f5d3d02018ff22e93447e919fc7f43774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-26T21:23:32.026 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 26 21:23:31 smithi094 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-26T21:23:32.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T21:23:32.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:32.315 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T21:23:32.315 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T21:23:32.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.z 2024-04-26T21:23:32.752 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:32 smithi204 systemd[1]: Stopping Ceph mgr.z for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:33.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:32 smithi204 podman[68773]: 2024-04-26 21:23:32.938084795 +0000 UTC m=+0.193569311 container died 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:23:33.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:33 smithi204 podman[68773]: 2024-04-26 21:23:33.385024334 +0000 UTC m=+0.640508875 container cleanup 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:23:33.502 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:33 smithi204 bash[68773]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z 2024-04-26T21:23:34.002 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:33 smithi204 podman[68786]: 2024-04-26 21:23:33.644640034 +0000 UTC m=+0.686795146 container remove 879f6c2ad24e9e994342bd5561f57a26e520ef0b4b1907bc6bb34295ddb6cf0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T21:23:34.002 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Apr 26 21:23:33 smithi204 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-26T21:23:34.139 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T21:23:34.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:34.249 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T21:23:34.249 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T21:23:34.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.0 2024-04-26T21:23:34.734 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:34 smithi027 systemd[1]: Stopping Ceph osd.0 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:35.235 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0[42443]: 2024-04-26T21:23:34.879+0000 7f017fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:35.235 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0[42443]: 2024-04-26T21:23:34.879+0000 7f017fe00700 -1 osd.0 104 *** Got signal Terminated *** 2024-04-26T21:23:35.235 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0[42443]: 2024-04-26T21:23:34.879+0000 7f017fe00700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:23:40.790 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:40 smithi027 podman[86875]: 2024-04-26 21:23:40.48549204 +0000 UTC m=+5.750576846 container died d757af214d3921c6413988499c97e3583596e9350ed013cebd90f3b755f910e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:23:41.136 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:40 smithi027 podman[86875]: 2024-04-26 21:23:40.904868187 +0000 UTC m=+6.169953030 container cleanup d757af214d3921c6413988499c97e3583596e9350ed013cebd90f3b755f910e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:23:41.136 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:40 smithi027 bash[86875]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0 2024-04-26T21:23:41.484 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:41 smithi027 podman[86896]: 2024-04-26 21:23:41.134581907 +0000 UTC m=+0.628823774 container remove d757af214d3921c6413988499c97e3583596e9350ed013cebd90f3b755f910e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T21:23:42.330 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[86982]: 2024-04-26 21:23:41.923163847 +0000 UTC m=+0.038953123 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:23:42.330 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[86982]: 2024-04-26 21:23:42.032440204 +0000 UTC m=+0.148229517 container create 645ff6ab50600d1dd7f28ecc3754e013b05123210f069f4f794e74610f07107b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:23:42.735 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[86982]: 2024-04-26 21:23:42.328110929 +0000 UTC m=+0.443900248 container init 645ff6ab50600d1dd7f28ecc3754e013b05123210f069f4f794e74610f07107b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:23:42.735 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[86982]: 2024-04-26 21:23:42.340381824 +0000 UTC m=+0.456171110 container start 645ff6ab50600d1dd7f28ecc3754e013b05123210f069f4f794e74610f07107b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:23:42.735 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[86982]: 2024-04-26 21:23:42.418125189 +0000 UTC m=+0.533914472 container attach 645ff6ab50600d1dd7f28ecc3754e013b05123210f069f4f794e74610f07107b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:23:43.184 INFO:journalctl@ceph.osd.0.smithi027.stdout:Apr 26 21:23:42 smithi027 podman[87007]: 2024-04-26 21:23:42.875536436 +0000 UTC m=+0.036029481 container died 645ff6ab50600d1dd7f28ecc3754e013b05123210f069f4f794e74610f07107b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T21:23:43.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T21:23:43.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:43.680 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T21:23:43.680 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T21:23:43.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.1 2024-04-26T21:23:44.026 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:43 smithi094 systemd[1]: Stopping Ceph osd.1 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:44.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1[35873]: 2024-04-26T21:23:44.284+0000 7f515a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:44.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1[35873]: 2024-04-26T21:23:44.284+0000 7f515a200700 -1 osd.1 104 *** Got signal Terminated *** 2024-04-26T21:23:44.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1[35873]: 2024-04-26T21:23:44.284+0000 7f515a200700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:23:50.091 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:49 smithi094 podman[67946]: 2024-04-26 21:23:49.791627138 +0000 UTC m=+5.673013849 container died 69db734a0b9a4305c9fcf182d5ccb480150c19b16e74e5ba3be821453bbe0bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:23:50.405 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:50 smithi094 podman[67946]: 2024-04-26 21:23:50.192991294 +0000 UTC m=+6.074378027 container cleanup 69db734a0b9a4305c9fcf182d5ccb480150c19b16e74e5ba3be821453bbe0bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:23:50.405 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:50 smithi094 bash[67946]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1 2024-04-26T21:23:50.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:50 smithi094 podman[67964]: 2024-04-26 21:23:50.402345879 +0000 UTC m=+0.601366680 container remove 69db734a0b9a4305c9fcf182d5ccb480150c19b16e74e5ba3be821453bbe0bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:23:51.492 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.09750421 +0000 UTC m=+0.033021451 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:23:51.492 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.228805815 +0000 UTC m=+0.164323093 container create 52de6e59893d63b67d6a33e45337bca12e38f311b251544edb343248333c170b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:23:51.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.490614307 +0000 UTC m=+0.426131591 container init 52de6e59893d63b67d6a33e45337bca12e38f311b251544edb343248333c170b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:23:51.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.502293006 +0000 UTC m=+0.437810278 container start 52de6e59893d63b67d6a33e45337bca12e38f311b251544edb343248333c170b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T21:23:51.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.589421593 +0000 UTC m=+0.524938845 container attach 52de6e59893d63b67d6a33e45337bca12e38f311b251544edb343248333c170b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T21:23:52.276 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 26 21:23:51 smithi094 podman[68053]: 2024-04-26 21:23:51.939510683 +0000 UTC m=+0.875027925 container died 52de6e59893d63b67d6a33e45337bca12e38f311b251544edb343248333c170b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:23:52.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T21:23:52.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:23:52.773 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T21:23:52.773 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T21:23:52.774 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.2 2024-04-26T21:23:53.252 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:52 smithi204 systemd[1]: Stopping Ceph osd.2 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:23:53.614 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2[37203]: 2024-04-26T21:23:53.361+0000 7f78cd600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:23:53.614 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2[37203]: 2024-04-26T21:23:53.361+0000 7f78cd600700 -1 osd.2 104 *** Got signal Terminated *** 2024-04-26T21:23:53.614 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2[37203]: 2024-04-26T21:23:53.361+0000 7f78cd600700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:23:59.205 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:58 smithi204 podman[68896]: 2024-04-26 21:23:58.882673395 +0000 UTC m=+5.663331625 container died b2fe0c47cd3f833edb8f15403947b314fc68fea0a35f167308b2fb06ef570300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:23:59.502 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:59 smithi204 podman[68896]: 2024-04-26 21:23:59.314140164 +0000 UTC m=+6.094798462 container cleanup b2fe0c47cd3f833edb8f15403947b314fc68fea0a35f167308b2fb06ef570300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:23:59.502 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:59 smithi204 bash[68896]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2 2024-04-26T21:24:00.002 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:23:59 smithi204 podman[68926]: 2024-04-26 21:23:59.590372346 +0000 UTC m=+0.694614456 container remove b2fe0c47cd3f833edb8f15403947b314fc68fea0a35f167308b2fb06ef570300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:00.753 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:00 smithi204 podman[69014]: 2024-04-26 21:24:00.335175195 +0000 UTC m=+0.034608601 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:00.753 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:00 smithi204 podman[69014]: 2024-04-26 21:24:00.441407314 +0000 UTC m=+0.140840786 container create 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:24:00.753 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:00 smithi204 podman[69014]: 2024-04-26 21:24:00.661622763 +0000 UTC m=+0.361056215 container init 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:24:00.753 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:00 smithi204 podman[69014]: 2024-04-26 21:24:00.673315456 +0000 UTC m=+0.372748884 container start 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T21:24:01.132 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:00 smithi204 podman[69014]: 2024-04-26 21:24:00.766454101 +0000 UTC m=+0.465887589 container attach 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:24:01.132 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:00.912+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:01.407 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:01 smithi204 podman[69014]: 2024-04-26 21:24:01.131803315 +0000 UTC m=+0.831236757 container died 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:24:01.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:01.221+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:01.662 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:01.405+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:01.913 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:01 smithi204 podman[69039]: 2024-04-26 21:24:01.872902306 +0000 UTC m=+0.722645225 container remove 28a04c996d81d36e8773db0c28a356ef70ac568574d3fae352d93142c9fb001f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T21:24:01.913 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:01 smithi204 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-26T21:24:01.913 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:01 smithi204 systemd[1]: Stopped Ceph osd.2 for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:24:01.913 INFO:journalctl@ceph.osd.2.smithi204.stdout:Apr 26 21:24:01 smithi204 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.2.service: Consumed 1min 21.866s CPU time. 2024-04-26T21:24:01.916 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T21:24:02.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:02.027 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T21:24:02.028 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T21:24:02.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.3 2024-04-26T21:24:02.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:01.929+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:02.275 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:02.003+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:02.485 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:02 smithi027 systemd[1]: Stopping Ceph osd.3 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:02.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:02.218+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:02.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:02.390+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:02.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:02.540+0000 7fb62e200700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:38.274740+0000 front 2024-04-26T21:23:38.274825+0000 (oldest deadline 2024-04-26T21:24:01.774915+0000) 2024-04-26T21:24:02.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:02.614+0000 7fb637a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:02.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:02.614+0000 7fb637a00700 -1 osd.3 104 *** Got signal Terminated *** 2024-04-26T21:24:02.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:02.614+0000 7fb637a00700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:03.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:02.974+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:02.974+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:03.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:03.030+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:03.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:03.239+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:03.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:03.411+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:03.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:03 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:03.546+0000 7fb62e200700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:38.274740+0000 front 2024-04-26T21:23:38.274825+0000 (oldest deadline 2024-04-26T21:24:01.774915+0000) 2024-04-26T21:24:04.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:03.984+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:03 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:03.974+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:04.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:04.041+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:04.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:04.269+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:04.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:04.394+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:04.985 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:04 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:04.568+0000 7fb62e200700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:38.274740+0000 front 2024-04-26T21:23:38.274825+0000 (oldest deadline 2024-04-26T21:24:01.774915+0000) 2024-04-26T21:24:05.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:05.022+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:05.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:04.955+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:05.486 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:05.058+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:05.486 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:05.275+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:05.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:05.401+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:05.984 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:05 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:05.558+0000 7fb62e200700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:38.274740+0000 front 2024-04-26T21:23:38.274825+0000 (oldest deadline 2024-04-26T21:24:01.774915+0000) 2024-04-26T21:24:06.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:05.988+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:06.275 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:05.942+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:06.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:06.054+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:06.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:06.239+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:06.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:06.377+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:06.984 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:06 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3[49352]: 2024-04-26T21:24:06.585+0000 7fb62e200700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:38.274740+0000 front 2024-04-26T21:23:38.274825+0000 (oldest deadline 2024-04-26T21:24:01.774915+0000) 2024-04-26T21:24:07.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:06.980+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:07.275 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:06.962+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:07.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:07.033+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:07.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:07 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:07.263+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:07.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:07.353+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:07.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:08.309 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:08.026+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:08.355 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:08.078+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:08.355 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:08 smithi027 podman[87095]: 2024-04-26 21:24:08.107088776 +0000 UTC m=+5.677427709 container died 31241fb78cda5ee004b7123631a102e7f7eb4f855c508aec24b958a8cb220d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T21:24:08.356 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:08 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:08.308+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:08.735 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:08 smithi027 podman[87095]: 2024-04-26 21:24:08.449473843 +0000 UTC m=+6.019812798 container cleanup 31241fb78cda5ee004b7123631a102e7f7eb4f855c508aec24b958a8cb220d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:24:08.735 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:08 smithi027 bash[87095]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3 2024-04-26T21:24:08.735 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:08 smithi027 podman[87117]: 2024-04-26 21:24:08.600426747 +0000 UTC m=+0.480625149 container remove 31241fb78cda5ee004b7123631a102e7f7eb4f855c508aec24b958a8cb220d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:08.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:08.307+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:09.290 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:09.024+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:09.451 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:09.077+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:09.451 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:09 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:09.259+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:09.525 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:09.025+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:09.735 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:09 smithi027 podman[87207]: 2024-04-26 21:24:09.356877327 +0000 UTC m=+0.036211639 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:09.735 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:09 smithi027 podman[87207]: 2024-04-26 21:24:09.51836035 +0000 UTC m=+0.197694712 container create fab7d0000597bc047603f286df2fa1a3886c108cce33f1e20c24f3833f600be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:24:09.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:09.288+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:10.226 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:09 smithi027 podman[87207]: 2024-04-26 21:24:09.823387017 +0000 UTC m=+0.502721373 container init fab7d0000597bc047603f286df2fa1a3886c108cce33f1e20c24f3833f600be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T21:24:10.226 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:09 smithi027 podman[87207]: 2024-04-26 21:24:09.835055031 +0000 UTC m=+0.514389413 container start fab7d0000597bc047603f286df2fa1a3886c108cce33f1e20c24f3833f600be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:24:10.226 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:09 smithi027 podman[87207]: 2024-04-26 21:24:09.904223081 +0000 UTC m=+0.583557412 container attach fab7d0000597bc047603f286df2fa1a3886c108cce33f1e20c24f3833f600be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T21:24:10.226 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:10.053+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:10.485 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 26 21:24:10 smithi027 podman[87233]: 2024-04-26 21:24:10.371426298 +0000 UTC m=+0.039293262 container died fab7d0000597bc047603f286df2fa1a3886c108cce33f1e20c24f3833f600be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T21:24:10.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:10 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:10.224+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:10.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:10.036+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:10.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:10.283+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:10.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:10.283+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:10.525 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:10.065+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:11.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T21:24:11.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:11.253 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T21:24:11.253 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T21:24:11.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.4 2024-04-26T21:24:11.275 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:11.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:11.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:11.273+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:11.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:11.274+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:11.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:11.048+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:11.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:11 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:11.048+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:11.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:11.053+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:11.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:11.290+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:11.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:11.290+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:11.776 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:11 smithi094 systemd[1]: Stopping Ceph osd.4 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:12.006+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:12.006+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:12.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:11.895+0000 7fdc45a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:12.277 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:11.895+0000 7fdc45a00700 -1 osd.4 104 *** Got signal Terminated *** 2024-04-26T21:24:12.277 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:11.895+0000 7fdc45a00700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:12.277 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:11 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:11.977+0000 7fdc41200700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.850146+0000 front 2024-04-26T21:23:48.850384+0000 (oldest deadline 2024-04-26T21:24:11.750320+0000) 2024-04-26T21:24:12.315 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:12.046+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:12.315 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:12.196+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:12.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:12.265+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:12.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:12.265+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:12.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:12.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:12.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:12 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:12.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:12.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:12.313+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:12.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:12.313+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:12.960 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:12.553+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:13.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:12.958+0000 7fdc41200700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.850146+0000 front 2024-04-26T21:23:48.850384+0000 (oldest deadline 2024-04-26T21:24:11.750320+0000) 2024-04-26T21:24:13.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:12.993+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:13.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:12.993+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:13.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:13.272+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:13.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:13.272+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:13.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:13.042+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:13.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:13.042+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:13.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:13.150+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:13.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:13.096+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:13.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:13.315+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:13.503 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:13.315+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:13.960 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:13 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:13.567+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:14.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:13 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:13.959+0000 7fdc41200700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.850146+0000 front 2024-04-26T21:23:48.850384+0000 (oldest deadline 2024-04-26T21:24:11.750320+0000) 2024-04-26T21:24:14.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:13 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:13.983+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:14.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:13 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:13.983+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:14.285 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:13.997+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:14.285 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:13 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:13.997+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:14.354 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:14.133+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:14.354 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:14.099+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:14.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:14 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:14.284+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:14.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:14 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:14.284+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:14.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:14.352+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:14.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:14.352+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:14.937 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:14.596+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:15.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:14.963+0000 7fdc41200700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.850146+0000 front 2024-04-26T21:23:48.850384+0000 (oldest deadline 2024-04-26T21:24:11.750320+0000) 2024-04-26T21:24:15.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:14.935+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:15.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:14 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:14.935+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:15.299 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:15.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:15.299 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:15.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:15.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:15.160+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:15.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:15.311+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:15.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:15.311+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:15.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:15.101+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:15.503 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:15.101+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:15.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:15.298+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:15.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:15 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:15.298+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:15.932 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:15 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:15.637+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:16.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:15 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4[42586]: 2024-04-26T21:24:15.930+0000 7fdc41200700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.850146+0000 front 2024-04-26T21:23:48.850384+0000 (oldest deadline 2024-04-26T21:24:11.750320+0000) 2024-04-26T21:24:16.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:15 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:15.934+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:16.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:15 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:15.934+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:16.295 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:16.041+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:16.295 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:16.041+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:16.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:16.118+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:16.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:16.262+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:16.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:16.262+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:16.503 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:16.075+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:16.503 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:16.075+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:16.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:16.294+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:16.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:16 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:16.294+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:16.933 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:16 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:16.616+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:17.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:16 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:16.931+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:17.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:16 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:16.931+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:17.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:17.004+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:17.341 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:17.004+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:17.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:17.145+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:17.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:17.062+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:17.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:17.062+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:17.503 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:17.282+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:17.503 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:17.282+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:17.579 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:17 smithi094 podman[68170]: 2024-04-26 21:24:17.29576559 +0000 UTC m=+5.577735353 container died a0150fd4f3d89ad6b1ff1b7fa2b1c47adea9427318e221264110bf187035ed03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:17.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:17.339+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:17.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:17 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:17.339+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:17.874 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:17 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:17.580+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:17.875 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:17 smithi094 podman[68170]: 2024-04-26 21:24:17.688692213 +0000 UTC m=+5.970662000 container cleanup a0150fd4f3d89ad6b1ff1b7fa2b1c47adea9427318e221264110bf187035ed03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T21:24:17.875 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:17 smithi094 bash[68170]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4 2024-04-26T21:24:18.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:17 smithi094 podman[68187]: 2024-04-26 21:24:17.873169998 +0000 UTC m=+0.560499638 container remove a0150fd4f3d89ad6b1ff1b7fa2b1c47adea9427318e221264110bf187035ed03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:24:18.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:17 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:17.894+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:18.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:17 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:17.894+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:18.300 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:18.000+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:18.300 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:18.000+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:18.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:18.123+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:18.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:18.028+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:18.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:18.028+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:18.503 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:18.242+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:18.503 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:18.242+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:18.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:18.298+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:18.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:18 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:18.298+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:18.939 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:18.590+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:18.940 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:18 smithi094 podman[68274]: 2024-04-26 21:24:18.531381798 +0000 UTC m=+0.034094798 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:18.940 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:18 smithi094 podman[68274]: 2024-04-26 21:24:18.64291353 +0000 UTC m=+0.145626552 container create daa66ec5eb4d3f21383bbd18b4b07e62e56a7b3b0cd0b35f14776a1bbf702b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:19.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:18.938+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:19.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:18 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:18.938+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:19.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:18 smithi094 podman[68274]: 2024-04-26 21:24:18.937715677 +0000 UTC m=+0.440428738 container init daa66ec5eb4d3f21383bbd18b4b07e62e56a7b3b0cd0b35f14776a1bbf702b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:19.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:18 smithi094 podman[68274]: 2024-04-26 21:24:18.950321306 +0000 UTC m=+0.453034320 container start daa66ec5eb4d3f21383bbd18b4b07e62e56a7b3b0cd0b35f14776a1bbf702b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:19.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:19 smithi094 podman[68274]: 2024-04-26 21:24:19.036882098 +0000 UTC m=+0.539595166 container attach daa66ec5eb4d3f21383bbd18b4b07e62e56a7b3b0cd0b35f14776a1bbf702b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:24:19.332 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:19.020+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:19.332 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:19.020+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:19.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:19.101+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:19.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:19.222+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:19.502 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:19.222+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:19.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:19.073+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:19.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:19.073+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:19.728 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:19 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:19.590+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:19.728 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 26 21:24:19 smithi094 podman[68274]: 2024-04-26 21:24:19.389897197 +0000 UTC m=+0.892610183 container died daa66ec5eb4d3f21383bbd18b4b07e62e56a7b3b0cd0b35f14776a1bbf702b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T21:24:19.734 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:19.330+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:19.734 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:19 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:19.330+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:20.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:19 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:19.923+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:20.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:19 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:19.923+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:20.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:19 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:19.923+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:20.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T21:24:20.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:20.252 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T21:24:20.252 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T21:24:20.252 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.5 2024-04-26T21:24:20.331 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:20.262+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:20.331 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:20.262+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:20.331 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:20.129+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:20.331 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:20.129+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:20.332 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:20.036+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:20.332 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:20.036+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:20.380 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:20.050+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:20.380 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:20.050+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:20.380 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:20.050+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:20.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:20.377+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:20.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:20.377+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:20.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:20 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:20.377+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:20.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 systemd[1]: Stopping Ceph osd.5 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:20.941 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:20.621+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:20.941 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:20.621+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:21.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:21.093+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:21.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:21.093+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:21.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:21.065+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:21.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:21.065+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:21.253 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:20.856+0000 7f4eefa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:21.253 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:20.856+0000 7f4eefa00700 -1 osd.5 104 *** Got signal Terminated *** 2024-04-26T21:24:21.253 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:20.856+0000 7f4eefa00700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:21.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:20.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:21.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:20.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:21.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:20 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:20.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:21.356 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.008+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:21.356 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.008+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:21.356 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.008+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:21.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:21.354+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:21.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:21.354+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:21.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:21.354+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:21.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:21.294+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:21.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:21 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:21.294+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:21.907 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:21 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:21.576+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:21.907 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:21 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:21.576+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:22.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:21 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:21.905+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:22.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:21 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:21.905+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:22.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:21 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:21.905+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:22.326 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:22.089+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:22.326 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:22.089+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:22.326 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:22.028+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:22.326 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:22.028+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:22.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.988+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:22.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.988+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:22.336 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:21 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:21.988+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:22.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:22.334+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:22.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:22.334+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:22.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:22.334+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:22.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:22.324+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:22.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:22.324+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:22.947 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:22 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:22.569+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:22.948 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:22 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:22.569+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:23.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:22.942+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:23.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:22.942+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:23.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:22 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:22.942+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:23.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:23.047+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:23.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:23.047+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:23.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:22.993+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:23.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:22.993+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:23.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:22 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:22.993+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:23.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:22 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:22.946+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:23.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:22 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:22.946+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:23.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:22 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:22.946+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:23.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:23.288+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:23.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:23.288+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:23.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:23.288+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:23.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:23.290+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:23.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:23.290+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:23.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:23 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:23.570+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:23.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:23 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:23.570+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:24.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:23.949+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:24.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:23.949+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:24.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:23 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:23.949+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:24.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:24.086+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:24.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:24.086+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:24.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:23.956+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:24.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:23.956+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:24.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:23 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:23.956+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:24.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:23 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:23.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:24.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:23 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:23.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:24.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:23 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:23.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:24.634 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:24.319+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:24.635 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:24.319+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:24.635 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:24.319+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.117432+0000 front 2024-04-26T21:23:58.117620+0000 (oldest deadline 2024-04-26T21:24:23.417402+0000) 2024-04-26T21:24:24.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:24.326+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:24.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:24.326+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:24.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:24.326+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:24.891 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:24 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:24.599+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:24.891 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:24 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:24.599+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:25.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:24.948+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:25.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:24.948+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:25.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:24 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:24.948+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:25.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:25.097+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:25.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:25.097+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:25.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:24.934+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:25.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:24.934+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:25.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:24 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:24.934+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:25.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:24 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:24.889+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:25.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:24 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:24.889+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:25.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:24 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:24.889+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:25.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:25.366+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:25.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:25.366+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:25.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:25.366+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:25.753 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:25.288+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:34.910373+0000 front 2024-04-26T21:23:34.910220+0000 (oldest deadline 2024-04-26T21:24:00.810343+0000) 2024-04-26T21:24:25.754 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:25.288+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:49.012951+0000 front 2024-04-26T21:23:49.013051+0000 (oldest deadline 2024-04-26T21:24:10.112934+0000) 2024-04-26T21:24:25.754 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5[44246]: 2024-04-26T21:24:25.288+0000 7f4ee6200700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.117432+0000 front 2024-04-26T21:23:58.117620+0000 (oldest deadline 2024-04-26T21:24:23.417402+0000) 2024-04-26T21:24:25.892 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:25.595+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:25.893 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:25.595+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:26.132 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:25.931+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:26.133 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:25.931+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:26.133 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:25 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:25.931+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:26.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:25.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:26.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:25.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:26.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:25.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:26.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:25.891+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:26.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:25.891+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:26.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:25 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:25.891+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:26.475 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:26.130+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:26.476 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:26.130+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:26.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:26 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:26.325+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:26.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:26 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:26.325+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:26.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:26 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:26.325+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:26.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:26 smithi204 podman[69120]: 2024-04-26 21:24:26.475177557 +0000 UTC m=+5.788657330 container died 4ef1861eb721ff2440171e8c3e7e256a5adc6722c5b3075272d61fe57d7a2c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T21:24:26.899 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:26 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:26.601+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:26.899 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:26 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:26.601+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:27.095 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:26 smithi204 podman[69120]: 2024-04-26 21:24:26.917600418 +0000 UTC m=+6.231080235 container cleanup 4ef1861eb721ff2440171e8c3e7e256a5adc6722c5b3075272d61fe57d7a2c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:27.095 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:26 smithi204 bash[69120]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5 2024-04-26T21:24:27.095 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:26.920+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:27.095 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:26.920+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:27.096 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:26 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:26.920+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:27.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:26 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:26.898+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:27.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:26 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:26.898+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:27.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:26 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:26.898+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:27.324 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.002+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:27.324 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.002+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:27.324 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.002+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:27.501 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:27 smithi204 podman[69153]: 2024-04-26 21:24:27.260640243 +0000 UTC m=+0.777806414 container remove 4ef1861eb721ff2440171e8c3e7e256a5adc6722c5b3075272d61fe57d7a2c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:24:27.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:27 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:27.092+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:27.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:27 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:27.092+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:27.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:27.322+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:27.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:27.322+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:27.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:27.322+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:28.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:27 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:27.630+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:28.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:27 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:27.630+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:28.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:27 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:27.848+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:28.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:27 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:27.848+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:28.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:27 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:27.848+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:28.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.978+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:28.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.978+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:28.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:27 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:27.978+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:28.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:28 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:28.073+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:28.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:28 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:28.073+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:28.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:27 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:27.910+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:28.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:27 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:27.910+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:28.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:27 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:27.910+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:28.253 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:27.968373023 +0000 UTC m=+0.036411006 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:28.253 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:28.086393419 +0000 UTC m=+0.154431443 container create 7c512200213bc50525afae0e5ba29a456202d0c6588e2092ad78143a1602888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T21:24:28.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:28.314+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:28.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:28.314+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:28.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:28.314+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:28.751 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:28.366765393 +0000 UTC m=+0.434803453 container init 7c512200213bc50525afae0e5ba29a456202d0c6588e2092ad78143a1602888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:28.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:28.378241781 +0000 UTC m=+0.446279775 container start 7c512200213bc50525afae0e5ba29a456202d0c6588e2092ad78143a1602888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T21:24:28.752 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:28.438836877 +0000 UTC m=+0.506874938 container attach 7c512200213bc50525afae0e5ba29a456202d0c6588e2092ad78143a1602888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T21:24:29.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:28.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:29.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:28.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:29.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:28.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:29.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:28.830+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:29.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:28.830+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:29.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:28 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:28.830+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:29.109 INFO:journalctl@ceph.osd.5.smithi204.stdout:Apr 26 21:24:28 smithi204 podman[69240]: 2024-04-26 21:24:28.827712827 +0000 UTC m=+0.895750878 container died 7c512200213bc50525afae0e5ba29a456202d0c6588e2092ad78143a1602888c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:24:29.109 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:28 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:28.894+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:29.109 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:28 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:28.894+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:29.109 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:28 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:28.894+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:29.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:28.960+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:29.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:28.960+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:29.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:28.960+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:29.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:29 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:29.107+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:29.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:29 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:29.107+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:29.611 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T21:24:29.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:29.727 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T21:24:29.727 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T21:24:29.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.6 2024-04-26T21:24:29.736 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:29.355+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:29.736 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:29.355+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:29.736 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:29.355+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:29.620+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:29.620+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:29.620+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:29.827+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:29.827+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:30.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:29 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:29.827+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:30.235 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:29 smithi027 systemd[1]: Stopping Ceph osd.6 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:30.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:29.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:30.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:29.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:30.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:29 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:29.971+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:30.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:30 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:30.108+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:30.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:30 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:30.108+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:30.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:29 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:29.898+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:30.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:29 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:29.898+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:30.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:29 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:29.898+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:30.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.292+0000 7f4ced800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:30.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.292+0000 7f4ced800700 -1 osd.6 104 *** Got signal Terminated *** 2024-04-26T21:24:30.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.292+0000 7f4ced800700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:30.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.311+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:30.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.311+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:30.736 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:30.311+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:31.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:30 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:30.851+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:31.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:30 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:30.851+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:31.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:30 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:30.851+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:30.581+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:30.581+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:30.581+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:30.807+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:30.807+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:31.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:30.807+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:31.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:30 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:30.807+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:31.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:30.970+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:31.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:30.970+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:31.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:30.970+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:31.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:30 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:30.970+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:31.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:31.070+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:31.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:31.070+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:31.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:31.070+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:31.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:31.335+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:31.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:31.335+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:31.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:31 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:31.335+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:31.852 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:31.590+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:31.852 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:31.590+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:31.852 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:31.590+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:32.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:32.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:32.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:32.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:32.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:32.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:32.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:31.818+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:32.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:31.818+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:32.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:31 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:31.818+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:32.269 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:32.016+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:32.269 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:32.016+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:32.269 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:32.016+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:32.269 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:32.016+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:32.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:31.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:32.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:31.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:32.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:31.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:32.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:31 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:31.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:32.622 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:32.363+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:32.622 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:32.363+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:32.622 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:32.363+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:32.622 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:32 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:32.363+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.096866+0000 front 2024-04-26T21:24:06.097384+0000 (oldest deadline 2024-04-26T21:24:31.397196+0000) 2024-04-26T21:24:32.896 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:32.622+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:32.897 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:32.622+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:32.897 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:32.622+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:33.098 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:32.777+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:33.098 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:32.777+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:33.098 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:32 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:32.777+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:33.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:33.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:33.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:33.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:33.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:33.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:33.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:33.005+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:33.273 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:32.895+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:33.273 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:32.895+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:33.273 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:32.895+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:33.273 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:32 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:32.895+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:33.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:33.096+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:33.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:33.096+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:33.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:33.096+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:33.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:33.379+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:33.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:33.379+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:33.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:33.379+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:33.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:33 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:33.379+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.096866+0000 front 2024-04-26T21:24:06.097384+0000 (oldest deadline 2024-04-26T21:24:31.397196+0000) 2024-04-26T21:24:33.858 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:33.596+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:33.858 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:33.596+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:33.858 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:33.596+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:34.107 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:33.800+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:34.107 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:33.800+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:34.107 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:33 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:33.800+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:34.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:34.027+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:34.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:34.027+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:34.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:34.027+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:34.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:34.027+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:34.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:33.856+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:34.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:33.856+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:34.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:33.856+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:34.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:33 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:33.856+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:34.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:34.106+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:34.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:34.106+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:34.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:34.106+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:34.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:34.411+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.387432+0000 front 2024-04-26T21:23:39.388064+0000 (oldest deadline 2024-04-26T21:24:01.087438+0000) 2024-04-26T21:24:34.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:34.411+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.789404+0000 front 2024-04-26T21:23:45.789295+0000 (oldest deadline 2024-04-26T21:24:10.489551+0000) 2024-04-26T21:24:34.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:34.411+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:55.092802+0000 front 2024-04-26T21:23:55.093231+0000 (oldest deadline 2024-04-26T21:24:19.792662+0000) 2024-04-26T21:24:34.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6[56292]: 2024-04-26T21:24:34.411+0000 7f4ce4000700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.096866+0000 front 2024-04-26T21:24:06.097384+0000 (oldest deadline 2024-04-26T21:24:31.397196+0000) 2024-04-26T21:24:34.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:34.621+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:34.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:34.621+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:34.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:34.621+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:35.094 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:34.757+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:35.094 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:34.757+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:35.095 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:34 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:34.757+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:35.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:34.872+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:35.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:34.872+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:35.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:34.872+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:35.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:34 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:34.872+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:35.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:35 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:35.011+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:35.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:35 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:35.011+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:35.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:35 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:35.011+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:35.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:35 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:35.011+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:35.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:35.092+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:35.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:35.092+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:35.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:35.092+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:36.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:35.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:36.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:35.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:36.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:35.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:36.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:35.850+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:36.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:35.659+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:36.027 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:35.659+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:36.027 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:35 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:35.659+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:36.061 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:35.808+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:36.061 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:35.808+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:36.062 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:35 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:35.808+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:36.184 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:36 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:36.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:36.184 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:36 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:36.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:36.184 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:36 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:36.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:36.184 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:36 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:36.046+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:36.184 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:35 smithi027 podman[87310]: 2024-04-26 21:24:35.890154857 +0000 UTC m=+5.749739897 container died 89a6762de6c1ad152c7d6b67d0f9fcf6b55142614eaa86255f70f29a3ef3b2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T21:24:36.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:36 smithi027 podman[87310]: 2024-04-26 21:24:36.318996704 +0000 UTC m=+6.178581744 container cleanup 89a6762de6c1ad152c7d6b67d0f9fcf6b55142614eaa86255f70f29a3ef3b2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:24:36.485 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:36 smithi027 bash[87310]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6 2024-04-26T21:24:36.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:36.059+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:36.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:36.059+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:36.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:36.059+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:36.648+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:36.648+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:36.648+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:36.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:36.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:36.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:37.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:36 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:36.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:37.049 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:36 smithi027 podman[87342]: 2024-04-26 21:24:36.736798816 +0000 UTC m=+0.836302883 container remove 89a6762de6c1ad152c7d6b67d0f9fcf6b55142614eaa86255f70f29a3ef3b2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:37.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:37.045+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:37.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:37.045+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:37.253 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:37.045+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:37.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:36.843+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:37.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:36.843+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:37.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:36 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:36.843+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:37.484 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:37 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:37.047+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:37.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:37 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:37.047+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:37.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:37 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:37.047+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:37.485 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:37 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:37.047+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:37.967 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:37 smithi027 podman[87428]: 2024-04-26 21:24:37.662546097 +0000 UTC m=+0.123398102 container create 231a35449fc7b67565d6af0afcc3545ed2a6c4e660cecdadcbed79244816491c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T21:24:37.967 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:37 smithi027 podman[87428]: 2024-04-26 21:24:37.57690782 +0000 UTC m=+0.037759802 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:37.967 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:37 smithi027 podman[87428]: 2024-04-26 21:24:37.966172947 +0000 UTC m=+0.427024962 container init 231a35449fc7b67565d6af0afcc3545ed2a6c4e660cecdadcbed79244816491c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:24:38.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:37.643+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:38.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:37.643+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:38.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:37.643+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:38.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:37.835+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:38.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:37.835+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:38.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:37.835+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:38.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:37 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:37.835+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:38.074 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:37.813+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:38.074 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:37.813+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:38.074 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:37 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:37.813+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.009+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.009+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.009+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.009+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:37 smithi027 podman[87428]: 2024-04-26 21:24:37.979554705 +0000 UTC m=+0.440406720 container start 231a35449fc7b67565d6af0afcc3545ed2a6c4e660cecdadcbed79244816491c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:38.235 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:38 smithi027 podman[87428]: 2024-04-26 21:24:38.048358894 +0000 UTC m=+0.509210959 container attach 231a35449fc7b67565d6af0afcc3545ed2a6c4e660cecdadcbed79244816491c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:24:38.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:38.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:38.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:38.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:38.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:38.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:38.735 INFO:journalctl@ceph.osd.6.smithi027.stdout:Apr 26 21:24:38 smithi027 podman[87428]: 2024-04-26 21:24:38.450717001 +0000 UTC m=+0.911569001 container died 231a35449fc7b67565d6af0afcc3545ed2a6c4e660cecdadcbed79244816491c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:38.877 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:38.623+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:38.877 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:38.623+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:38.877 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:38.623+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:39.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:38.816+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:39.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:38.816+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:39.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:38.816+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:39.102 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:38 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:38.816+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:39.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.963+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:39.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.963+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:39.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.963+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:39.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.963+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:39.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:38 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:38.963+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:39.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:38.876+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:39.278 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:38.876+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:39.278 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:38.876+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:39.278 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:38 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:38.876+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:39.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T21:24:39.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:39.101+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:39.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:39.101+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:39.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:39.101+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:39.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:39.101+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:39.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:39.515 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T21:24:39.516 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T21:24:39.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.7 2024-04-26T21:24:39.893 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:39.588+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:39.893 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:39.588+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:39.893 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:39.588+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:39.893 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:39 smithi094 systemd[1]: Stopping Ceph osd.7 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:40.081 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:39.772+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:40.081 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:39.772+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:40.081 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:39.772+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:40.081 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:39 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:39.772+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:40.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:39 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:39.918+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:40.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:39 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:39.918+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:40.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:39 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:39.918+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:40.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:39 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:39.918+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:40.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:39 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:39.918+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:40.276 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.058+0000 7fb97ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:40.276 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.058+0000 7fb97ea00700 -1 osd.7 104 *** Got signal Terminated *** 2024-04-26T21:24:40.276 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.058+0000 7fb97ea00700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:40.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:39.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:40.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:39.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:40.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:39.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:40.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:39 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:39.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:40.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:40.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:40.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:40.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:40.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:40.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:40.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:40.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:40.861 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.600+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:40.861 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.600+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:40.861 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.600+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:40.861 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:40.600+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:13.009835+0000 front 2024-04-26T21:24:13.010093+0000 (oldest deadline 2024-04-26T21:24:40.010969+0000) 2024-04-26T21:24:41.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:40.907+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:41.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:40.907+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:41.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:40.907+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:41.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:40.907+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:41.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:40.907+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:41.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:41.033+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:41.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:41.033+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:41.033+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:41.033+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:40.788+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:40.788+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:40.788+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:41.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:40 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:40.788+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:41.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:40.859+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:41.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:40.859+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:41.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:40.859+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:41.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:40 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:40.859+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:42.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:41.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:42.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:41.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:42.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:41.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:42.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:41.641+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:13.009835+0000 front 2024-04-26T21:24:13.010093+0000 (oldest deadline 2024-04-26T21:24:40.010969+0000) 2024-04-26T21:24:42.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:41.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:42.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:41.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:42.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:41.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:42.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:41.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:42.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:41 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:41.878+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:42.067 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:41.759+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:42.067 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:41.759+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:42.067 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:41.759+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:42.067 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:41 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:41.759+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:42.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:41 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:41.882+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:42.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:41 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:41.882+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:42.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:41 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:41.882+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:42.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:41 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:41.882+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:42.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:41 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:41.882+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:42.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:42.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:42.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:42.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:42.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:42.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:42.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:42.066+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:43.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:42.709+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:43.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:42.709+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:43.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:42.709+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:43.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:42 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:42.709+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:43.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:42.684+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:43.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:42.685+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:43.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:42.685+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:43.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:42.685+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:13.009835+0000 front 2024-04-26T21:24:13.010093+0000 (oldest deadline 2024-04-26T21:24:40.010969+0000) 2024-04-26T21:24:43.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:42.860+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:43.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:42.860+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:43.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:42.860+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:43.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:42.860+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:43.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:42 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:42.860+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:43.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:42 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:42.883+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:43.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:42 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:42.883+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:43.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:42 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:42.883+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:43.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:42 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:42.883+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:43.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:42 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:42.883+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:43.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:43.065+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:43.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:43.065+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:43.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:43.065+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:43.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:43.065+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:43.937 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:43.674+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:43.937 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:43.674+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:43.674+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:43.674+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:13.009835+0000 front 2024-04-26T21:24:13.010093+0000 (oldest deadline 2024-04-26T21:24:40.010969+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:43.897+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:43.897+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:43.897+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:43.897+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:43.938 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:43 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:43.897+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:44.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:43.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:44.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:43.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:44.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:43.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:44.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:43 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:43.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:44.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:43 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:43.919+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:44.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:43 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:43.919+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:44.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:43 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:43.919+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:44.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:43 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:43.919+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:44.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:43 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:43.919+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:44.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:44.094+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:44.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:44.094+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:44.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:44.094+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:44.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:44.094+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:44.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:44.669+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:47.598939+0000 front 2024-04-26T21:23:47.599134+0000 (oldest deadline 2024-04-26T21:24:12.299253+0000) 2024-04-26T21:24:44.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:44.669+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:56.303645+0000 front 2024-04-26T21:23:56.303433+0000 (oldest deadline 2024-04-26T21:24:20.403438+0000) 2024-04-26T21:24:44.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:44.669+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:06.707513+0000 front 2024-04-26T21:24:06.707812+0000 (oldest deadline 2024-04-26T21:24:28.308230+0000) 2024-04-26T21:24:44.921 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7[49409]: 2024-04-26T21:24:44.669+0000 7fb97a200700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:13.009835+0000 front 2024-04-26T21:24:13.010093+0000 (oldest deadline 2024-04-26T21:24:40.010969+0000) 2024-04-26T21:24:45.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:44.720+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:45.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:44.720+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:45.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:44.720+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:45.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:44 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:44.720+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:45.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:44 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:44.888+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:45.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:44 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:44.888+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:45.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:44 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:44.888+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:45.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:44 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:44.888+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:45.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:44 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:44.888+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:45.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:44.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:45.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:44.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:45.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:44.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:45.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:44.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:45.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:44 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:44.919+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:45.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:45.071+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:45.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:45.071+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:45.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:45.071+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:45.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:45.071+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:45.879 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:45 smithi094 podman[68390]: 2024-04-26 21:24:45.608313207 +0000 UTC m=+5.691781665 container died 12274264afb7dd17e19b1d429f1613d434b2b2cae56cddd61fae5578753c9601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T21:24:46.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:45.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:46.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:45.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:46.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:45.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:46.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:45 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:45.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:45 smithi094 podman[68390]: 2024-04-26 21:24:45.991932696 +0000 UTC m=+6.075401158 container cleanup 12274264afb7dd17e19b1d429f1613d434b2b2cae56cddd61fae5578753c9601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:45 smithi094 bash[68390]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:45.877+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:45.877+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:45.877+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:45.877+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:46.204 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:45 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:45.877+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:46.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:45 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:45.900+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:46.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:45 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:45.900+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:46.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:45 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:45.900+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:46.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:45 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:45.900+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:46.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:45 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:45.900+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:46.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:46.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:46.517 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:46.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:46.517 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:46.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:46.517 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:46.080+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:46.525 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:46 smithi094 podman[68410]: 2024-04-26 21:24:46.201553014 +0000 UTC m=+0.578689561 container remove 12274264afb7dd17e19b1d429f1613d434b2b2cae56cddd61fae5578753c9601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:24:47.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:46.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:47.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:46.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:47.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:46.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:47.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:46 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:46.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:47.161 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:46.906+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:47.161 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:46.906+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:47.161 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:46.906+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:47.161 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:46.906+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:47.162 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:46.906+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:47.162 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:46 smithi094 podman[68498]: 2024-04-26 21:24:46.874400851 +0000 UTC m=+0.035889562 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:47.162 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:46 smithi094 podman[68498]: 2024-04-26 21:24:46.994128354 +0000 UTC m=+0.155617101 container create 45c3e0f7a3d08bfdd20a0c897b42d57a63ec7f3b7f458b9833224869a70fafa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:24:47.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:46 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:46.910+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:47.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:46 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:46.910+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:47.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:46 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:46.910+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:47.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:46 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:46.910+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:47.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:46 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:46.910+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:47.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:47.090+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:47.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:47.090+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:47.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:47.090+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:47.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:47.090+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:47.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:47 smithi094 podman[68498]: 2024-04-26 21:24:47.264250509 +0000 UTC m=+0.425739263 container init 45c3e0f7a3d08bfdd20a0c897b42d57a63ec7f3b7f458b9833224869a70fafa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:24:47.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:47 smithi094 podman[68498]: 2024-04-26 21:24:47.275665671 +0000 UTC m=+0.437154424 container start 45c3e0f7a3d08bfdd20a0c897b42d57a63ec7f3b7f458b9833224869a70fafa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:24:47.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:47 smithi094 podman[68498]: 2024-04-26 21:24:47.363004142 +0000 UTC m=+0.524492935 container attach 45c3e0f7a3d08bfdd20a0c897b42d57a63ec7f3b7f458b9833224869a70fafa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-26T21:24:48.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:47.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:48.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:47.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:48.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:47.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:48.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:47 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:47.726+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 26 21:24:47 smithi094 podman[68498]: 2024-04-26 21:24:47.725263185 +0000 UTC m=+0.886751939 container died 45c3e0f7a3d08bfdd20a0c897b42d57a63ec7f3b7f458b9833224869a70fafa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:47.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:47.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:47.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:47.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:48.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:47 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:47.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:48.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:47 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:47.951+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:48.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:47 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:47.951+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:48.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:47 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:47.951+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:48.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:47 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:47.951+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:48.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:47 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:47.951+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:48.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T21:24:48.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:48.058+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:48.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:48.058+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:48.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:48.058+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:48.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:48.058+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:48.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:48.572 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T21:24:48.572 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T21:24:48.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.8 2024-04-26T21:24:49.001 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:48 smithi204 systemd[1]: Stopping Ceph osd.8 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:49.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:48.684+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:49.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:48.684+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:49.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:48.684+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:49.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:48.684+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:49.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:48 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:48.684+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:49.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:48 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:48.912+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:49.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:48.904+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:49.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:48.904+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:49.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:48.904+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:49.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:48.904+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:49.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:48 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:48.904+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:49.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.128+0000 7f50aba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.128+0000 7f50aba00700 -1 osd.8 104 *** Got signal Terminated *** 2024-04-26T21:24:49.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:49.128+0000 7f50aba00700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:50.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:49.711+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:50.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:49.711+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:50.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:49.711+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:50.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:49.711+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:50.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:49 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:49.711+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:50.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:49 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:49.904+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:50.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:49 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:49.917+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:50.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:50.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:50.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:50.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:50.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:50.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:50.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:50.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:50.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:50.072+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.507680+0000 front 2024-04-26T21:24:23.507972+0000 (oldest deadline 2024-04-26T21:24:49.407963+0000) 2024-04-26T21:24:51.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:50.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:51.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:50.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:51.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:50.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:51.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:50.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:51.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:50 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:50.749+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:51.175 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:51.175 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:51.175 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:51.175 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:51.175 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:51.176 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:50 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:50.892+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:51.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:50 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:50.933+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:51.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:51.052+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:51.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:51.052+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:51.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:51.052+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:51.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:51.052+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:51.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:51.052+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.507680+0000 front 2024-04-26T21:24:23.507972+0000 (oldest deadline 2024-04-26T21:24:49.407963+0000) 2024-04-26T21:24:52.092 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:51.796+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:52.093 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:51.796+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:52.093 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:51.796+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:52.093 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:51.796+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:52.093 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:51 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:51.796+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:52.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:51 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:51.943+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:52.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:51 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:51.939+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:52.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:52.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:52.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:52.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:52.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:52.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:52.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:52.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:52.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:52.091+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.507680+0000 front 2024-04-26T21:24:23.507972+0000 (oldest deadline 2024-04-26T21:24:49.407963+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:53.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:52 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:52.928+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:53.055+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:53.055+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:53.055+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:53.055+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:53.055+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.507680+0000 front 2024-04-26T21:24:23.507972+0000 (oldest deadline 2024-04-26T21:24:49.407963+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:52.806+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:53.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:52.806+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:53.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:52.806+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:53.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:52.806+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:53.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:52 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:52.806+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:53.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:52.986+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:53.943 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:53.824+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:53.943 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:53.824+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:53.944 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:53.824+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:53.944 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:53.824+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:53.944 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:53 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:53.824+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:54.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:54.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:54.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:54.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:54.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:54.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:53 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:53.901+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:54.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:54.104+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.392801+0000 front 2024-04-26T21:23:48.392729+0000 (oldest deadline 2024-04-26T21:24:11.292915+0000) 2024-04-26T21:24:54.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:54.104+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:57.095999+0000 front 2024-04-26T21:23:57.096277+0000 (oldest deadline 2024-04-26T21:24:19.996341+0000) 2024-04-26T21:24:54.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:54.104+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:04.402010+0000 front 2024-04-26T21:24:04.401260+0000 (oldest deadline 2024-04-26T21:24:30.301218+0000) 2024-04-26T21:24:54.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:54.104+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.605300+0000 front 2024-04-26T21:24:16.605226+0000 (oldest deadline 2024-04-26T21:24:38.904977+0000) 2024-04-26T21:24:54.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8[50745]: 2024-04-26T21:24:54.104+0000 7f50a2200700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.507680+0000 front 2024-04-26T21:24:23.507972+0000 (oldest deadline 2024-04-26T21:24:49.407963+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:54.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:53 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:53.992+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:54.919 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:54 smithi204 podman[69344]: 2024-04-26 21:24:54.628454621 +0000 UTC m=+5.630684647 container died 96c99154793ef52838e76f9e4f3a7f3da26c23da3f3c283d5bde2098615a9ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:24:54.920 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:54.852+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:54.920 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:54.852+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:54.920 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:54.852+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:54.920 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:54.852+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:54.920 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:54 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:54.852+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:55.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:54 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:54.864+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:55.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:55 smithi204 podman[69344]: 2024-04-26 21:24:55.011694495 +0000 UTC m=+6.013924555 container cleanup 96c99154793ef52838e76f9e4f3a7f3da26c23da3f3c283d5bde2098615a9ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:24:55.252 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:55 smithi204 bash[69344]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8 2024-04-26T21:24:55.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:55.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:55.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:55.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:55.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:55.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:55 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:55.026+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:55.751 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:55 smithi204 podman[69373]: 2024-04-26 21:24:55.280121122 +0000 UTC m=+0.644465855 container remove 96c99154793ef52838e76f9e4f3a7f3da26c23da3f3c283d5bde2098615a9ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:24:56.072 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:55 smithi204 podman[69461]: 2024-04-26 21:24:55.806663067 +0000 UTC m=+0.036976575 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:24:56.072 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:55 smithi204 podman[69461]: 2024-04-26 21:24:55.905437903 +0000 UTC m=+0.135751448 container create cd7c3423e76f33b89f5f5a1cda9de20d0a045d83bdf34a16a9b33dea60307aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:24:56.072 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:55 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:55.814+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:56.072 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:55 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:55.814+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:56.072 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:55 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:55.814+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:56.073 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:55 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:55.814+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:56.073 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:55 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:55.814+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:56.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:55 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:55.816+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:56.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:56 smithi204 podman[69461]: 2024-04-26 21:24:56.16751432 +0000 UTC m=+0.397827908 container init cd7c3423e76f33b89f5f5a1cda9de20d0a045d83bdf34a16a9b33dea60307aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T21:24:56.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:56 smithi204 podman[69461]: 2024-04-26 21:24:56.18004672 +0000 UTC m=+0.410360237 container start cd7c3423e76f33b89f5f5a1cda9de20d0a045d83bdf34a16a9b33dea60307aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T21:24:56.503 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:56 smithi204 podman[69461]: 2024-04-26 21:24:56.257706554 +0000 UTC m=+0.488020065 container attach cd7c3423e76f33b89f5f5a1cda9de20d0a045d83bdf34a16a9b33dea60307aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:56.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:56.048+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:56.985 INFO:journalctl@ceph.osd.8.smithi204.stdout:Apr 26 21:24:56 smithi204 podman[69461]: 2024-04-26 21:24:56.628607124 +0000 UTC m=+0.858920699 container died cd7c3423e76f33b89f5f5a1cda9de20d0a045d83bdf34a16a9b33dea60307aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T21:24:56.985 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:56 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:56.825+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:56.985 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:56 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:56.825+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:56.985 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:56 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:56.825+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:56.985 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:56 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:56.825+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:56.986 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:56 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:56.825+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:57.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:56 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:56.826+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:57.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.002+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:57.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T21:24:57.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:24:57.424 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T21:24:57.424 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T21:24:57.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.9 2024-04-26T21:24:57.877 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 systemd[1]: Stopping Ceph osd.9 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:24:58.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:58.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:57 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:57.875+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.028+0000 7fa40ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.028+0000 7fa40ea00700 -1 osd.9 104 *** Got signal Terminated *** 2024-04-26T21:24:58.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.028+0000 7fa40ea00700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:24:58.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:57 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:57.832+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:58.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:57 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:57.832+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:58.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:57 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:57.832+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:58.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:57 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:57.832+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:58.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:57 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:57.832+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:58.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:57 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:57.971+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:24:59.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:24:59.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:24:59.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:24:59.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:24:59.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:24:59.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:58 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:58.837+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:24:59.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:58 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:58.783+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:24:59.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:58 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:58.783+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:24:59.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:58 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:58.783+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:24:59.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:58 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:58.783+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:24:59.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:58 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:58.783+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:24:59.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:58 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:58.980+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:25:00.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:24:59 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:24:59.829+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:25:00.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:00.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:00.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:00.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:00.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:00.253 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:24:59 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:24:59.760+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:00.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:00.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:00.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:00.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:00.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:00.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:00.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:24:59 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:24:59.960+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:01.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:01.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:01.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:01.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:01.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:01.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:00 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:00.727+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:25:01.137 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:00 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:00.877+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:01.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:01.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:00.976+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:02.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:02.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:02.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:02.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:02.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:02.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:01 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:01.682+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:25:02.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:01 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:01.896+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:02.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:02.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:01 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:01.982+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:03.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:03.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:03.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:03.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:03.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:03.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:02 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:02.725+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.738502+0000 front 2024-04-26T21:23:37.738976+0000 (oldest deadline 2024-04-26T21:24:02.438574+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:45.242087+0000 front 2024-04-26T21:23:45.242254+0000 (oldest deadline 2024-04-26T21:24:10.541991+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:54.644140+0000 front 2024-04-26T21:23:54.644210+0000 (oldest deadline 2024-04-26T21:24:19.944418+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:05.147857+0000 front 2024-04-26T21:24:05.148240+0000 (oldest deadline 2024-04-26T21:24:30.447795+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:14.550362+0000 front 2024-04-26T21:24:14.550431+0000 (oldest deadline 2024-04-26T21:24:38.050498+0000) 2024-04-26T21:25:03.236 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:02 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9[63157]: 2024-04-26T21:25:02.890+0000 7fa405200700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.654846+0000 front 2024-04-26T21:24:25.654737+0000 (oldest deadline 2024-04-26T21:24:48.554702+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:03.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:02 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:02.972+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:03.912 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:03 smithi027 podman[87531]: 2024-04-26 21:25:03.661330479 +0000 UTC m=+5.765570975 container died 0936803af54a865b7225015c896caf91e80fe9a37e6aa34933ec54ac6f334b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T21:25:04.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:04.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:04.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:04.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:04.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:04.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:03 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:03.738+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:04.197 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:04 smithi027 podman[87531]: 2024-04-26 21:25:04.004062707 +0000 UTC m=+6.108303247 container cleanup 0936803af54a865b7225015c896caf91e80fe9a37e6aa34933ec54ac6f334b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:25:04.197 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:04 smithi027 bash[87531]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9 2024-04-26T21:25:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:04.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:04.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:04.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:04 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:04.020+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:04.484 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:04 smithi027 podman[87549]: 2024-04-26 21:25:04.196905387 +0000 UTC m=+0.523831746 container remove 0936803af54a865b7225015c896caf91e80fe9a37e6aa34933ec54ac6f334b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:04.953 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:04.699+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:05.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:04 smithi027 podman[87637]: 2024-04-26 21:25:04.887497523 +0000 UTC m=+0.036692950 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:25:05.235 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:05 smithi027 podman[87637]: 2024-04-26 21:25:05.039382614 +0000 UTC m=+0.188578034 container create 7a15524eba283cbfdc91d6e976a285b8b004ca33111c574b6e02c46abb2ded05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:05.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:05.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:05 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:05.043+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:05.735 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:05 smithi027 podman[87637]: 2024-04-26 21:25:05.384957928 +0000 UTC m=+0.534153394 container init 7a15524eba283cbfdc91d6e976a285b8b004ca33111c574b6e02c46abb2ded05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T21:25:05.735 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:05 smithi027 podman[87637]: 2024-04-26 21:25:05.39694412 +0000 UTC m=+0.546139586 container start 7a15524eba283cbfdc91d6e976a285b8b004ca33111c574b6e02c46abb2ded05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:25:05.735 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:05 smithi027 podman[87637]: 2024-04-26 21:25:05.475454147 +0000 UTC m=+0.624649659 container attach 7a15524eba283cbfdc91d6e976a285b8b004ca33111c574b6e02c46abb2ded05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T21:25:06.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:06.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:06.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:06.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:06.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:06.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:05 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:05.676+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:06.142 INFO:journalctl@ceph.osd.9.smithi027.stdout:Apr 26 21:25:05 smithi027 podman[87637]: 2024-04-26 21:25:05.873291181 +0000 UTC m=+1.022486604 container died 7a15524eba283cbfdc91d6e976a285b8b004ca33111c574b6e02c46abb2ded05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:25:06.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:06.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:06.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:06.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:06.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:06.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:06.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.019+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:06.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T21:25:06.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:06.605 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T21:25:06.605 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T21:25:06.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.10 2024-04-26T21:25:06.972 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 systemd[1]: Stopping Ceph osd.10 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:07.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:07.029 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:06 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:06.680+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:07.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:06 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:06.970+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:07.158+0000 7f12eae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:07.158+0000 7f12eae00700 -1 osd.10 104 *** Got signal Terminated *** 2024-04-26T21:25:07.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:07 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:07.158+0000 7f12eae00700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:25:08.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:08.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:08.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:08.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:08.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:08.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:08.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:07 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:07.642+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:08.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:08.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:08.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.011+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:09.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:09.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:09.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:09.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:09.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:09.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:09.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:08.606+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:09.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:08 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:08.989+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:10.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:10.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:10.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:10.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:10.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:10.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:10.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:09 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:09.624+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:10.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:10.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:09 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:09.961+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:11.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:11.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:11.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:11.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:11.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:11.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:11.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:10 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:10.576+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:11.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:11.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:10 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:10.968+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:12.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:12.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:12.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:12.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:12.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:12.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:12.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:11 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:11.618+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:39.577632+0000 front 2024-04-26T21:23:39.577815+0000 (oldest deadline 2024-04-26T21:24:01.877880+0000) 2024-04-26T21:25:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.782429+0000 front 2024-04-26T21:23:48.782672+0000 (oldest deadline 2024-04-26T21:24:11.082105+0000) 2024-04-26T21:25:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:53.983950+0000 front 2024-04-26T21:23:53.983822+0000 (oldest deadline 2024-04-26T21:24:19.884093+0000) 2024-04-26T21:25:12.276 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-04-26T21:24:02.186557+0000 front 2024-04-26T21:24:02.186749+0000 (oldest deadline 2024-04-26T21:24:30.387722+0000) 2024-04-26T21:25:12.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.190412+0000 front 2024-04-26T21:24:16.190528+0000 (oldest deadline 2024-04-26T21:24:41.490556+0000) 2024-04-26T21:25:12.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:25.592311+0000 front 2024-04-26T21:24:25.592373+0000 (oldest deadline 2024-04-26T21:24:49.092604+0000) 2024-04-26T21:25:12.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10[56533]: 2024-04-26T21:25:12.001+0000 7f12e6600700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.394941+0000 front 2024-04-26T21:24:34.394674+0000 (oldest deadline 2024-04-26T21:24:59.094632+0000) 2024-04-26T21:25:13.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:13.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:13.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:13.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:13.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:13.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:13.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:12.586+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:13.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:12 smithi094 podman[68613]: 2024-04-26 21:25:12.760801048 +0000 UTC m=+5.740184476 container died 8f96df0dc6a3ed3bf39fd782944f8b2a7696af5b8025c4b1a851480bdf42ecb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:25:13.437 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:13 smithi094 podman[68613]: 2024-04-26 21:25:13.176162185 +0000 UTC m=+6.155545657 container cleanup 8f96df0dc6a3ed3bf39fd782944f8b2a7696af5b8025c4b1a851480bdf42ecb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T21:25:13.437 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:13 smithi094 bash[68613]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10 2024-04-26T21:25:13.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:13 smithi094 podman[68631]: 2024-04-26 21:25:13.435285483 +0000 UTC m=+0.655971431 container remove 8f96df0dc6a3ed3bf39fd782944f8b2a7696af5b8025c4b1a851480bdf42ecb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:25:14.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:14.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:14.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:14.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:14.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:14.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:14.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:13 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:13.582+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:14.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.137756252 +0000 UTC m=+0.035601736 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:25:14.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.25262747 +0000 UTC m=+0.150472955 container create 696f28ba7401259e5a0fa791b2b2519785ce86c333f7b8d393a59e8e730f2d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:25:14.993 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.531016432 +0000 UTC m=+0.428861944 container init 696f28ba7401259e5a0fa791b2b2519785ce86c333f7b8d393a59e8e730f2d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T21:25:14.993 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.542946141 +0000 UTC m=+0.440791676 container start 696f28ba7401259e5a0fa791b2b2519785ce86c333f7b8d393a59e8e730f2d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T21:25:14.993 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.621583269 +0000 UTC m=+0.519428824 container attach 696f28ba7401259e5a0fa791b2b2519785ce86c333f7b8d393a59e8e730f2d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:25:15.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:15.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:15.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:15.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:15.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:15.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:15.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:14 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:14.574+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:15.275 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 26 21:25:14 smithi094 podman[68718]: 2024-04-26 21:25:14.993046981 +0000 UTC m=+0.890892499 container died 696f28ba7401259e5a0fa791b2b2519785ce86c333f7b8d393a59e8e730f2d4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True) 2024-04-26T21:25:15.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T21:25:15.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:15.777 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T21:25:15.778 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T21:25:15.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.11 2024-04-26T21:25:15.854 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:15.854 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:15.854 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:15.854 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:15.854 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:15.855 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:15.855 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:15.553+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:16.252 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:15 smithi204 systemd[1]: Stopping Ceph osd.11 for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:16.754 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.404+0000 7fc142400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:16.754 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.404+0000 7fc142400700 -1 osd.11 104 *** Got signal Terminated *** 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.404+0000 7fc142400700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:16.755 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:16.508+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:18.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:18.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:17 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:17.554+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-04-26T21:24:53.838410+0000 front 2024-04-26T21:24:53.838360+0000 (oldest deadline 2024-04-26T21:25:16.738131+0000) 2024-04-26T21:25:19.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:19.003 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:18 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:18.526+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-04-26T21:24:53.838410+0000 front 2024-04-26T21:24:53.838360+0000 (oldest deadline 2024-04-26T21:25:16.738131+0000) 2024-04-26T21:25:19.752 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:19.752 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:19.752 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:19.752 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:19.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:19.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:19.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:19.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:19 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:19.489+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-04-26T21:24:53.838410+0000 front 2024-04-26T21:24:53.838360+0000 (oldest deadline 2024-04-26T21:25:16.738131+0000) 2024-04-26T21:25:20.752 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-04-26T21:23:37.913186+0000 front 2024-04-26T21:23:37.913084+0000 (oldest deadline 2024-04-26T21:24:00.813103+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-26T21:23:48.917129+0000 front 2024-04-26T21:23:48.917049+0000 (oldest deadline 2024-04-26T21:24:14.216766+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-04-26T21:23:58.219727+0000 front 2024-04-26T21:23:58.220079+0000 (oldest deadline 2024-04-26T21:24:22.319794+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-26T21:24:16.826123+0000 front 2024-04-26T21:24:16.825971+0000 (oldest deadline 2024-04-26T21:24:37.925698+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-04-26T21:24:23.728517+0000 front 2024-04-26T21:24:23.728607+0000 (oldest deadline 2024-04-26T21:24:48.428562+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-04-26T21:24:34.231549+0000 front 2024-04-26T21:24:34.231748+0000 (oldest deadline 2024-04-26T21:24:59.531396+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-26T21:24:44.634862+0000 front 2024-04-26T21:24:44.634742+0000 (oldest deadline 2024-04-26T21:25:05.734561+0000) 2024-04-26T21:25:20.753 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:20 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11[57422]: 2024-04-26T21:25:20.466+0000 7fc138c00700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-04-26T21:24:53.838410+0000 front 2024-04-26T21:24:53.838360+0000 (oldest deadline 2024-04-26T21:25:16.738131+0000) 2024-04-26T21:25:22.189 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:21 smithi204 podman[69566]: 2024-04-26 21:25:21.898667453 +0000 UTC m=+5.665991399 container died 4cb0fe0e018e522c6f5d739a2328bbf26c13f0fca5475fde797f598706b9432e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T21:25:22.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:22 smithi204 podman[69566]: 2024-04-26 21:25:22.306585307 +0000 UTC m=+6.073909269 container cleanup 4cb0fe0e018e522c6f5d739a2328bbf26c13f0fca5475fde797f598706b9432e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T21:25:22.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:22 smithi204 bash[69566]: ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11 2024-04-26T21:25:23.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:22 smithi204 podman[69585]: 2024-04-26 21:25:22.534810239 +0000 UTC m=+0.619474702 container remove 4cb0fe0e018e522c6f5d739a2328bbf26c13f0fca5475fde797f598706b9432e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:25:23.666 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:23 smithi204 podman[69673]: 2024-04-26 21:25:23.399957779 +0000 UTC m=+0.125516409 container create 9055a25a4fd5b444e08670cdd8a01cc160c92ba20df176bfb8c609f63806e86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T21:25:23.666 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:23 smithi204 podman[69673]: 2024-04-26 21:25:23.311346427 +0000 UTC m=+0.036905033 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:25:23.666 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:23 smithi204 podman[69673]: 2024-04-26 21:25:23.66406165 +0000 UTC m=+0.389620345 container init 9055a25a4fd5b444e08670cdd8a01cc160c92ba20df176bfb8c609f63806e86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T21:25:24.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:23 smithi204 podman[69673]: 2024-04-26 21:25:23.675616102 +0000 UTC m=+0.401174754 container start 9055a25a4fd5b444e08670cdd8a01cc160c92ba20df176bfb8c609f63806e86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:25:24.002 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:23 smithi204 podman[69673]: 2024-04-26 21:25:23.753370234 +0000 UTC m=+0.478928861 container attach 9055a25a4fd5b444e08670cdd8a01cc160c92ba20df176bfb8c609f63806e86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:25:24.502 INFO:journalctl@ceph.osd.11.smithi204.stdout:Apr 26 21:25:24 smithi204 podman[69673]: 2024-04-26 21:25:24.143991105 +0000 UTC m=+0.869549782 container died 9055a25a4fd5b444e08670cdd8a01cc160c92ba20df176bfb8c609f63806e86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T21:25:24.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T21:25:24.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:24.978 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T21:25:24.978 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T21:25:24.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.a 2024-04-26T21:25:25.485 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:25 smithi027 systemd[1]: Stopping Ceph mds.a for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:25.985 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: 2024-04-26T21:25:25.578+0000 7f2e01e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:25.985 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:25 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a[70028]: 2024-04-26T21:25:25.578+0000 7f2e01e00700 -1 mds.a *** got signal Terminated *** 2024-04-26T21:25:27.077 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:26 smithi027 podman[87740]: 2024-04-26 21:25:26.810643178 +0000 UTC m=+1.381023985 container died 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T21:25:27.382 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:27 smithi027 podman[87740]: 2024-04-26 21:25:27.213437606 +0000 UTC m=+1.783818363 container cleanup 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:25:27.382 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:27 smithi027 bash[87740]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a 2024-04-26T21:25:27.735 INFO:journalctl@ceph.mds.a.smithi027.stdout:Apr 26 21:25:27 smithi027 podman[87754]: 2024-04-26 21:25:27.380727894 +0000 UTC m=+0.558545929 container remove 9f9b1def5d51dbf3b90d91ad2894792084978c05a29de537d814caf483a403a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:25:27.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T21:25:28.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:28.016 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T21:25:28.017 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T21:25:28.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.d 2024-04-26T21:25:28.427 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:28 smithi027 systemd[1]: Stopping Ceph mds.d for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:28.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d[68968]: 2024-04-26T21:25:28.673+0000 7fbc37000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:28.735 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:28 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d[68968]: 2024-04-26T21:25:28.673+0000 7fbc37000700 -1 mds.d *** got signal Terminated *** 2024-04-26T21:25:32.772 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:32 smithi027 podman[87860]: 2024-04-26 21:25:32.49771073 +0000 UTC m=+3.979319889 container died fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T21:25:33.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:32 smithi027 podman[87860]: 2024-04-26 21:25:32.883727427 +0000 UTC m=+4.365336607 container cleanup fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:25:33.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:32 smithi027 bash[87860]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d 2024-04-26T21:25:33.485 INFO:journalctl@ceph.mds.d.smithi027.stdout:Apr 26 21:25:33 smithi027 podman[87884]: 2024-04-26 21:25:33.134946411 +0000 UTC m=+0.620076432 container remove fa27592029f208be2e0fdadac13d1499063fc608aec68eab81ba9d2ec07721d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T21:25:33.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T21:25:33.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:33.804 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T21:25:33.804 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T21:25:33.804 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.g 2024-04-26T21:25:34.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:33 smithi027 systemd[1]: Stopping Ceph mds.g for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:34.734 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: 2024-04-26T21:25:34.469+0000 7f77f6e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:34.735 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:34 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g[69639]: 2024-04-26T21:25:34.469+0000 7f77f6e00700 -1 mds.g *** got signal Terminated *** 2024-04-26T21:25:38.830 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:38 smithi027 podman[87990]: 2024-04-26 21:25:38.487274321 +0000 UTC m=+4.198767682 container died 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T21:25:39.234 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:38 smithi027 podman[87990]: 2024-04-26 21:25:38.828808865 +0000 UTC m=+4.540302239 container cleanup 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:25:39.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:38 smithi027 bash[87990]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g 2024-04-26T21:25:39.235 INFO:journalctl@ceph.mds.g.smithi027.stdout:Apr 26 21:25:39 smithi027 podman[88005]: 2024-04-26 21:25:39.004975169 +0000 UTC m=+0.497751580 container remove 2bb0e60da5ec1d5c69c8e7b1785a0a6f3e9cf811fb74a8cc607488d203f0eb47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:25:39.452 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T21:25:39.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:39.569 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T21:25:39.569 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T21:25:39.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.j 2024-04-26T21:25:39.984 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:39 smithi027 systemd[1]: Stopping Ceph mds.j for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:40.485 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j[68582]: 2024-04-26T21:25:40.171+0000 7fec66200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:40.485 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:40 smithi027 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j[68582]: 2024-04-26T21:25:40.171+0000 7fec66200700 -1 mds.j *** got signal Terminated *** 2024-04-26T21:25:44.485 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:44 smithi027 podman[88113]: 2024-04-26 21:25:44.19081484 +0000 UTC m=+4.134862317 container died 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:25:44.985 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:44 smithi027 podman[88113]: 2024-04-26 21:25:44.682906082 +0000 UTC m=+4.626953569 container cleanup 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:25:44.985 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:44 smithi027 bash[88113]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j 2024-04-26T21:25:45.371 INFO:journalctl@ceph.mds.j.smithi027.stdout:Apr 26 21:25:45 smithi027 podman[88127]: 2024-04-26 21:25:45.083904398 +0000 UTC m=+0.871046799 container remove 9333d71dbf2d3f646db91d73dec19dd49dc6d5719060deba9615e4c9547cd121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T21:25:45.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T21:25:45.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:45.534 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T21:25:45.534 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T21:25:45.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.b 2024-04-26T21:25:46.026 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:45 smithi094 systemd[1]: Stopping Ceph mds.b for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:46.431 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: 2024-04-26T21:25:46.147+0000 7f3481600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:46.431 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:46 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b[62173]: 2024-04-26T21:25:46.147+0000 7f3481600700 -1 mds.b *** got signal Terminated *** 2024-04-26T21:25:50.464 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:50 smithi094 podman[68824]: 2024-04-26 21:25:50.160101284 +0000 UTC m=+4.184532540 container died c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T21:25:50.776 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:50 smithi094 podman[68824]: 2024-04-26 21:25:50.574991649 +0000 UTC m=+4.599422931 container cleanup c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T21:25:50.776 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:50 smithi094 bash[68824]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b 2024-04-26T21:25:51.276 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 26 21:25:50 smithi094 podman[68849]: 2024-04-26 21:25:50.892743157 +0000 UTC m=+0.712461403 container remove c8825ae95623a5dab4320b5f784117d2b8e653869ae1a924c2489b58aaf8207a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:25:51.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T21:25:51.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:51.543 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T21:25:51.543 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T21:25:51.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.e 2024-04-26T21:25:52.026 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:51 smithi094 systemd[1]: Stopping Ceph mds.e for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:52.526 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: 2024-04-26T21:25:52.092+0000 7fd05d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:52.526 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:52 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e[61553]: 2024-04-26T21:25:52.092+0000 7fd05d800700 -1 mds.e *** got signal Terminated *** 2024-04-26T21:25:55.162 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:54 smithi094 podman[68955]: 2024-04-26 21:25:54.836195504 +0000 UTC m=+2.853458360 container died f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:25:55.480 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:55 smithi094 podman[68955]: 2024-04-26 21:25:55.286934556 +0000 UTC m=+3.304197405 container cleanup f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:25:55.481 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:55 smithi094 bash[68955]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e 2024-04-26T21:25:55.775 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 26 21:25:55 smithi094 podman[68968]: 2024-04-26 21:25:55.479355601 +0000 UTC m=+0.626448353 container remove f80392088d2f36bc042623bb88a6894388b030472f4b5e32ca48ab7311b2fa5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T21:25:55.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T21:25:56.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:25:56.095 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T21:25:56.095 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T21:25:56.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.h 2024-04-26T21:25:56.275 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:56 smithi094 systemd[1]: Stopping Ceph mds.h for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:25:57.025 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: 2024-04-26T21:25:56.621+0000 7ff9bde00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:25:57.026 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:56 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h[61861]: 2024-04-26T21:25:56.621+0000 7ff9bde00700 -1 mds.h *** got signal Terminated *** 2024-04-26T21:25:59.145 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:58 smithi094 podman[69070]: 2024-04-26 21:25:58.853097141 +0000 UTC m=+2.358480789 container died 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T21:25:59.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:59 smithi094 podman[69070]: 2024-04-26 21:25:59.265247406 +0000 UTC m=+2.770631061 container cleanup 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:25:59.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:59 smithi094 bash[69070]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h 2024-04-26T21:26:00.026 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 26 21:25:59 smithi094 podman[69084]: 2024-04-26 21:25:59.571763896 +0000 UTC m=+0.708681604 container remove 121b01ab36cca22698d5fcf3b3846255faf12016c855af886f8506cd37222513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T21:26:00.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T21:26:00.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:00.190 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T21:26:00.190 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T21:26:00.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.k 2024-04-26T21:26:00.525 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:00 smithi094 systemd[1]: Stopping Ceph mds.k for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:26:01.026 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: 2024-04-26T21:26:00.724+0000 7fe910400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:26:01.026 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:00 smithi094 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k[61251]: 2024-04-26T21:26:00.724+0000 7fe910400700 -1 mds.k *** got signal Terminated *** 2024-04-26T21:26:03.190 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:02 smithi094 podman[69190]: 2024-04-26 21:26:02.841063494 +0000 UTC m=+2.245248213 container died 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:26:03.526 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:03 smithi094 podman[69190]: 2024-04-26 21:26:03.310159414 +0000 UTC m=+2.714344156 container cleanup 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:26:03.526 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:03 smithi094 bash[69190]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k 2024-04-26T21:26:04.026 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 26 21:26:03 smithi094 podman[69204]: 2024-04-26 21:26:03.552613674 +0000 UTC m=+0.692271863 container remove 9ecf582fb8880498733b97bd8c6d3a3c1b289be26f1d6529cf38541cbba405c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:26:04.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T21:26:04.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:04.184 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T21:26:04.184 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T21:26:04.184 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.c 2024-04-26T21:26:04.714 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:04 smithi204 systemd[1]: Stopping Ceph mds.c for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:26:05.002 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: 2024-04-26T21:26:04.711+0000 7f1b49400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:26:05.002 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:04 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c[62740]: 2024-04-26T21:26:04.711+0000 7f1b49400700 -1 mds.c *** got signal Terminated *** 2024-04-26T21:26:07.153 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:06 smithi204 podman[69779]: 2024-04-26 21:26:06.874954092 +0000 UTC m=+2.302238062 container died 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:26:07.502 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:07 smithi204 podman[69779]: 2024-04-26 21:26:07.273024858 +0000 UTC m=+2.700308861 container cleanup 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:26:07.502 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:07 smithi204 bash[69779]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c 2024-04-26T21:26:07.969 INFO:journalctl@ceph.mds.c.smithi204.stdout:Apr 26 21:26:07 smithi204 podman[69804]: 2024-04-26 21:26:07.538565727 +0000 UTC m=+0.646991791 container remove 63a3836eb5336a3af260a373774b8fefc55a76133e44f06bec2c02c9aef845eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:26:08.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T21:26:08.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:08.110 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T21:26:08.110 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T21:26:08.110 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.f 2024-04-26T21:26:08.752 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:08 smithi204 systemd[1]: Stopping Ceph mds.f for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:26:09.252 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: 2024-04-26T21:26:08.750+0000 7fdfb2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:26:09.252 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:08 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f[62426]: 2024-04-26T21:26:08.750+0000 7fdfb2400700 -1 mds.f *** got signal Terminated *** 2024-04-26T21:26:11.116 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:10 smithi204 podman[69910]: 2024-04-26 21:26:10.835881421 +0000 UTC m=+2.214984182 container died 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:26:11.501 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 podman[69910]: 2024-04-26 21:26:11.218897181 +0000 UTC m=+2.597999942 container cleanup 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:26:11.502 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 bash[69910]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f 2024-04-26T21:26:11.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T21:26:11.861 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 podman[69922]: 2024-04-26 21:26:11.553511642 +0000 UTC m=+0.701644909 container remove 825fa59664a43b8612a0156a4763ff2372689c50f85812dfac569d1274b2459d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:26:11.862 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-26T21:26:11.862 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 systemd[1]: Stopped Ceph mds.f for a125c556-040f-11ef-bc93-c7b262605968. 2024-04-26T21:26:11.862 INFO:journalctl@ceph.mds.f.smithi204.stdout:Apr 26 21:26:11 smithi204 systemd[1]: ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.f.service: Consumed 5.724s CPU time. 2024-04-26T21:26:11.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:11.895 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T21:26:11.895 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T21:26:11.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.i 2024-04-26T21:26:12.251 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:12 smithi204 systemd[1]: Stopping Ceph mds.i for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:26:12.752 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: 2024-04-26T21:26:12.438+0000 7f4636800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:26:12.752 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:12 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i[63043]: 2024-04-26T21:26:12.438+0000 7f4636800700 -1 mds.i *** got signal Terminated *** 2024-04-26T21:26:15.128 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:14 smithi204 podman[70029]: 2024-04-26 21:26:14.835858202 +0000 UTC m=+2.516421855 container died b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:26:15.500 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:15 smithi204 podman[70029]: 2024-04-26 21:26:15.223092475 +0000 UTC m=+2.903656121 container cleanup b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T21:26:15.500 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:15 smithi204 bash[70029]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i 2024-04-26T21:26:15.750 INFO:journalctl@ceph.mds.i.smithi204.stdout:Apr 26 21:26:15 smithi204 podman[70044]: 2024-04-26 21:26:15.498999548 +0000 UTC m=+0.647672702 container remove b153d31e5de25f32f0220035ccd7b3983f29fd45e07cfd1eddae3b6528e06425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T21:26:15.765 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T21:26:15.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:15.868 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T21:26:15.868 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T21:26:15.868 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.l 2024-04-26T21:26:16.251 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:15 smithi204 systemd[1]: Stopping Ceph mds.l for a125c556-040f-11ef-bc93-c7b262605968... 2024-04-26T21:26:16.754 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: 2024-04-26T21:26:16.366+0000 7f3945800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:26:16.754 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:16 smithi204 ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l[62123]: 2024-04-26T21:26:16.366+0000 7f3945800700 -1 mds.l *** got signal Terminated *** 2024-04-26T21:26:19.140 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:18 smithi204 podman[70150]: 2024-04-26 21:26:18.873412156 +0000 UTC m=+2.627878952 container stop 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T21:26:19.140 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:18 smithi204 podman[70150]: 2024-04-26 21:26:18.899516225 +0000 UTC m=+2.653983012 container died 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:26:19.446 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:19 smithi204 podman[70150]: 2024-04-26 21:26:19.251714048 +0000 UTC m=+3.006180872 container cleanup 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T21:26:19.446 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:19 smithi204 bash[70150]: ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l 2024-04-26T21:26:19.751 INFO:journalctl@ceph.mds.l.smithi204.stdout:Apr 26 21:26:19 smithi204 podman[70164]: 2024-04-26 21:26:19.445477497 +0000 UTC m=+0.551049505 container remove 4e50074c1ca0cfaf76a0c7b67ee096bac9c65f0eef2cba24c790cea1526ca19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a125c556-040f-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:26:19.950 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a125c556-040f-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T21:26:20.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:26:20.050 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T21:26:20.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:26:20.385 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:25.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:26:26.151 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:30.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:26:31.144 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:36.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:26:36.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:26:36.382 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:26:36.447 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T21:26:36.450 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi027/crash 2024-04-26T21:26:36.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:26:36.556 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:26:36.556 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:26:36.561 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi094/crash 2024-04-26T21:26:36.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:26:36.662 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:26:36.662 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:26:36.667 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi204/crash 2024-04-26T21:26:36.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:26:36.793 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/a125c556-040f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:26:36.793 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:26:36.797 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T21:26:36.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T21:26:36.913 INFO:tasks.cephadm:Compressing logs... 2024-04-26T21:26:36.913 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:26:36.918 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:26:36.925 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:26:36.975 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:26:36.975 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-26T21:26:36.976 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:26:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:26:36.979 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:26:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T21:26:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T21:26:36.982 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T21:26:36.983 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:26:36.984 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T21:26:36.984 INFO:teuthology.orchestra.run.smithi027.stderr: 90.2% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:26:36.985 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T21:26:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T21:26:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.x.log: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:26:36.990 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.a.log: 94.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T21:26:36.990 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T21:26:36.992 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:26:36.995 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T21:26:36.995 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:26:36.995 INFO:teuthology.orchestra.run.smithi027.stderr: 84.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:26:36.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T21:26:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:26:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:26:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:26:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:26:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T21:26:37.005 INFO:teuthology.orchestra.run.smithi094.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:26:37.006 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T21:26:37.006 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T21:26:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.7.log: 87.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:26:37.007 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T21:26:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.b.log: 90.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:26:37.009 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T21:26:37.010 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:26:37.010 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:26:37.010 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T21:26:37.011 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:26:37.012 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T21:26:37.012 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T21:26:37.013 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T21:26:37.014 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:26:37.014 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:26:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T21:26:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:26:37.016 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T21:26:37.016 INFO:teuthology.orchestra.run.smithi204.stderr: 87.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:26:37.017 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T21:26:37.017 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T21:26:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T21:26:37.018 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T21:26:37.020 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T21:26:37.021 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.c.log: 90.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:26:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.e.log: 83.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log.gzgzip 2024-04-26T21:26:37.022 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:26:37.022 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T21:26:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T21:26:37.023 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:26:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T21:26:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T21:26:37.028 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:26:37.028 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T21:26:37.030 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T21:26:37.031 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:26:37.034 INFO:teuthology.orchestra.run.smithi027.stderr: 91.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:26:37.035 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.8.log: 92.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T21:26:37.038 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T21:26:37.061 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.k.log: 90.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:26:37.070 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.l.log: 90.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:26:37.080 INFO:teuthology.orchestra.run.smithi027.stderr: 94.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T21:26:37.089 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T21:26:37.094 INFO:teuthology.orchestra.run.smithi094.stderr: 92.7% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T21:26:37.096 INFO:teuthology.orchestra.run.smithi204.stderr: 94.5% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T21:26:37.103 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T21:26:37.112 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T21:26:37.112 INFO:teuthology.orchestra.run.smithi027.stderr: 92.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T21:26:37.122 INFO:teuthology.orchestra.run.smithi094.stderr: 92.7% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T21:26:37.141 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T21:26:37.354 INFO:teuthology.orchestra.run.smithi204.stderr: 91.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T21:26:37.390 INFO:teuthology.orchestra.run.smithi094.stderr: 92.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T21:26:37.409 INFO:teuthology.orchestra.run.smithi027.stderr: 90.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T21:26:37.437 INFO:teuthology.orchestra.run.smithi094.stderr: 92.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T21:26:37.885 INFO:teuthology.orchestra.run.smithi094.stderr: 92.7% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T21:26:37.946 INFO:teuthology.orchestra.run.smithi027.stderr: 90.3% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T21:26:37.956 INFO:teuthology.orchestra.run.smithi204.stderr: 92.7% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T21:26:38.029 INFO:teuthology.orchestra.run.smithi204.stderr: 92.7% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T21:26:38.034 INFO:teuthology.orchestra.run.smithi204.stderr: 93.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T21:26:38.177 INFO:teuthology.orchestra.run.smithi027.stderr: 93.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T21:26:38.217 INFO:teuthology.orchestra.run.smithi094.stderr: 92.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T21:26:38.268 INFO:teuthology.orchestra.run.smithi094.stderr: 92.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T21:26:38.328 INFO:teuthology.orchestra.run.smithi204.stderr: 92.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T21:26:38.333 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-04-26T21:26:38.333 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m1.353s 2024-04-26T21:26:38.333 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m4.509s 2024-04-26T21:26:38.333 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.482s 2024-04-26T21:26:38.370 INFO:teuthology.orchestra.run.smithi027.stderr: 92.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T21:26:38.373 INFO:teuthology.orchestra.run.smithi094.stderr: 93.0% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T21:26:38.378 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-26T21:26:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m1.406s 2024-04-26T21:26:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m5.283s 2024-04-26T21:26:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.511s 2024-04-26T21:26:38.436 INFO:teuthology.orchestra.run.smithi027.stderr: 92.8% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T21:26:38.460 INFO:teuthology.orchestra.run.smithi027.stderr: 92.9% -- replaced with /var/log/ceph/a125c556-040f-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T21:26:46.306 INFO:teuthology.orchestra.run.smithi027.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-26T21:26:46.311 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-26T21:26:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m9.360s 2024-04-26T21:26:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m15.123s 2024-04-26T21:26:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m1.298s 2024-04-26T21:26:46.313 INFO:tasks.cephadm:Archiving logs... 2024-04-26T21:26:46.313 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi027/log 2024-04-26T21:26:46.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:26:48.024 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi094/log 2024-04-26T21:26:48.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:26:48.831 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi204/log 2024-04-26T21:26:48.832 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:26:49.591 INFO:tasks.cephadm:Removing cluster... 2024-04-26T21:26:49.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force 2024-04-26T21:26:49.934 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:50.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force 2024-04-26T21:26:50.938 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:51.559 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a125c556-040f-11ef-bc93-c7b262605968 --force 2024-04-26T21:26:51.886 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: a125c556-040f-11ef-bc93-c7b262605968 2024-04-26T21:26:52.441 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T21:26:52.442 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:26:52.500 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:26:52.558 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:26:52.615 INFO:tasks.cephadm:Teardown complete 2024-04-26T21:26:52.615 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:26:52.628 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:26:52.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:26:52.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:26:52.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:26:52.803 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:26:52.814 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:26:52.814 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:26:52.822 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:26:52.830 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:26:52.872 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:26:52.875 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 7 377 30 +1187us[+1181us] +/- 45ms 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 29 -870us[ -870us] +/- 62ms 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -856us[ -863us] +/- 78ms 2024-04-26T21:26:52.883 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 7 377 89 +1201us[+1195us] +/- 44ms 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 26 -944us[ -944us] +/- 62ms 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -907us[ -907us] +/- 78ms 2024-04-26T21:26:52.885 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:26:52.894 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:26:52.895 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-04-26T21:26:52.895 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1159us[+1173us] +/- 45ms 2024-04-26T21:26:52.895 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -958us[ -944us] +/- 62ms 2024-04-26T21:26:52.895 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -893us[ -879us] +/- 78ms 2024-04-26T21:26:52.895 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:26:52.896 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:26:52.909 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:26:52.909 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:26:52.938 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:26:53.163 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:26:53.345 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:26:53.547 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:26:53.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:53.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:26:53.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:53.742 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:26:53.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:53.835 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:26:53.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:26:53.897 INFO:teuthology.task.internal:Duration was 2232.523719 seconds 2024-04-26T21:26:53.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:26:53.939 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:26:53.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:26:53.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:26:53.955 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:26:54.070 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:26:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:26:54.099 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:26:54.598 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:26:54.598 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-04-26T21:26:54.598 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:26:54.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T21:26:54.671 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:26:54.750 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-04-26T21:26:54.750 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:26:54.826 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:26:54.826 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:26:54.833 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:26:54.841 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:26:54.925 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:26:54.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:26:54.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:26:54.939 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:26:55.525 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:26:55.537 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:26:55.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:26:55.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:26:55.717 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:26:55.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:26:55.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:55.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:55.834 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:55.924 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-26T21:26:55.928 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-04-26T21:26:55.928 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-04-26T21:26:55.989 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:56.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:56.042 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:56.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:56.095 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:26:56.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:26:56.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:26:56.161 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:26:56.162 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi027 2024-04-26T21:26:56.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:26:56.278 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi094 2024-04-26T21:26:56.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:26:56.378 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675181/remote/smithi204 2024-04-26T21:26:56.378 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:26:56.476 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:26:56.477 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:26:56.485 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:26:56.492 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:26:56.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:26:56.562 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:26:56.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:26:56.612 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:26:56.612 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:26:56.620 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:26:56.628 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:26:56.674 INFO:teuthology.orchestra.run.smithi027.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:26 /home/ubuntu/cephtest 2024-04-26T21:26:56.679 INFO:teuthology.orchestra.run.smithi094.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:26 /home/ubuntu/cephtest 2024-04-26T21:26:56.685 INFO:teuthology.orchestra.run.smithi204.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:26 /home/ubuntu/cephtest 2024-04-26T21:26:56.688 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:26:56.701 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:26:56.782 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} duration: 2232.5237193107605 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:26:56.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:26:56.857 INFO:teuthology.run:pass